ai

Butterbase AI Gateway

The AI gateway provides chat, embeddings, model listing, configuration, usage reporting, and asynchronous video jobs. One umbrella tool: manage_ai.

1. Chat

manage_ai({
  action: "chat",
  app_id,
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user",   content: "What's RAG?" }
  ],
  model: "openai/gpt-4o-mini",     // optional — falls back to app's default
  temperature: 0.2,                // optional
  max_tokens: 500                  // optional
})

This action sets stream: false deliberately — agent tools don't stream. If you need partial-token deltas, drive the SDK's ai.chatStream(…) from inside a function or DO instead.

messages[].content can be a string or an array of content parts ({ type: "text", text }, { type: "image_url", image_url: {...} }, { type: "video_url", video_url: {...} }).

2. Embed

manage_ai({
  action: "embed",
  app_id,
  input: "hello world",            // or ["a", "b", "c"]
  model: "openai/text-embedding-3-small",   // optional
  encoding_format: "float"          // or "base64"
})

3. List models

manage_ai({ action: "list_models", app_id })
// → { models: [{ id, provider, capabilities: ["chat", "embed", ...], context_window, pricing }, ...] }

Use this to discover what the app can call — capabilities + context window matter when picking a model.

4. Configure

manage_ai({
  action: "update_config",
  app_id,
  config: {
    defaultModel: "openai/gpt-4o-mini",
    allowedModels: ["openai/gpt-4o-mini", "anthropic/claude-haiku-4-5"],
    maxTokensPerRequest: 4000,
    byokKey: "..." // optional — rotates the customer-supplied OpenRouter / Anthropic key
  }
})

• maxTokensPerRequest is server-clamped to 1–100000.
• allowedModels is a whitelist — empty means all models the provider exposes.
• Setting byokKey switches the app to route through that customer key. Clear it by passing byokKey: "" (returns to platform pool).

5. Usage

manage_ai({
  action: "get_usage",
  app_id,
  startDate: "2026-05-01",
  endDate:   "2026-05-31"
})

Returns aggregate token counts + cost. Useful for billing reconciliation, spending-cap diagnostics, and showing dashboards.

6. Common pitfalls

• Trying to stream from a tool — manage_ai is synchronous. Use the SDK inside a function for streamed deltas.
• Sending stream: true in the body — the tool ignores it; always wired to false.
• Hardcoding model — better to omit, let the app's defaultModel win, and surface that knob via update_config.
• Skipping list_models before suggesting one — model availability shifts; verify before recommending.

7. What this skill does NOT cover

• Streaming chat — use the SDK (ai.chatStream) inside a function or DO.
• Vector storage / retrieval — see butterbase-skills:rag-dev (RAG collections wrap embeddings + search together).
• AI in deployed functions — they import @butterbase/sdk and call client.ai.* directly; no MCP needed at runtime.

8. Decisions (typed classification)

For routing, classification, moderation or scoring, where code needs a choice / yes-no / score with probabilities, use manage_ai action: "decide" (SDK ai.decide) with a decision model (default typesafe/jev-1.13) instead of asking a chat model for JSON. It is cheaper, faster and returns typed probabilities. Full reference: butterbase_docs topic ai, "Decision models".

9. Video jobs and declared fictional references

Before video work, call butterbase_docs with topic: "ai" and discover supported models with manage_ai({ action: "list_models", app_id, modality: "video" }).

Preserve the returned job ID, show the reported stage, poll without issuing another submission, and display authenticated content URLs after completion. Use the caller's explicit fictional-character declaration only. Never silently add declarations or automatically label real people as fictional.

The declaration example below belongs in the REST body for POST /v1/:appId/videos/completions; poll GET /v1/:appId/videos/completions/:jobId. The currently released MCP submit_video wrapper does not forward reference_declarations or prepare_references, so do not put these fields into that tool call and assume preparation occurred. Use the documented REST surface for this opt-in contract; poll_video can read the resulting job by ID. Consult current docs before choosing a client surface; do not invent an SDK video helper.

Optional fictional character references

Eligible apps and models can declare specific image slots as AI-generated fictional characters:

{
  "input_images": ["https://images.example/fictional-character.png"],
  "reference_declarations": [
    {"path": "/input_images/0", "kind": "ai_generated_fictional_character"}
  ],
  "prepare_references": "on_rejection"
}

Add these fields to an ordinary video request. Declare only images the caller knows are AI-generated fictional characters. A declaration does not establish consent for a real person's likeness or guarantee that an image will pass provider validation. Do not infer a declaration from appearance or a prompt.

on_rejection is the default: prepare only after a verified eligible input rejection and retry once after settlement. upfront prepares all declared slots before submission. Canonical paths address image entries in input_images, frame_images, or input_references; use one canonical collection and avoid conflicting aliases. Up to eight distinct slots can be declared. Undeclared requests use ordinary video generation.

Submission returns the same stable job_id while the worker advances through preparing_references, submitting, polling, and settling. Polling reads status; it does not trigger generation. Completion includes authenticated content URLs and acknowledged charges. Unsupported app/model combinations return UNSUPPORTED_REFERENCE_PREPARATION. Changed image bytes, unavailable or rejected assets, and capacity exhaustion may fail preparation. A timeout with unknown provider acceptance remains pending for reconciliation; do not submit another job merely because polling is slow.

Preparation sends declared images to an external generation provider for storage and validation. Registration is not separately charged in this release; video generation uses the app's normal credits and settled usage. Source URLs and prompts are removed from preparation records after 24 hours. Provider copies are queued for deletion when the last retained job reference is released, subject to active jobs and unresolved operations; provider deletion can therefore take longer. App deletion preserves cleanup obligations and unsettled billing evidence. URL inputs have no storage-object identity: deleting the original file does not immediately delete its provider copy. Use stable storage objectId values in your own app and resolve fresh download URLs when loading UI.

There is no character-library dashboard. Preparation eligibility and resource limits are deployment-controlled. For troubleshooting, inspect the returned stage and sanitized error, verify the declaration and unchanged image bytes, and contact the operator for unresolved acceptance or cleanup.

If a docs/butterbase/00-state.md exists in the working directory, prefer invoking via /butterbase-skills:journey-ai so the journey orchestrator stays in sync.