dust-llm

Adding Support for a New LLM Model

This skill guides you through adding a newly released LLM to the model_constructors + llms stack (the endpoint-class router). It replaces the legacy lib/api/llm/clients/* router, which no longer exists.

Mental model

A model reaches production through three stacked layers. Add the new model to each:

1. Model config (front/types/assistant/models/*) — the legacy ModelConfigurationType describing the model (context, vision, reasoning efforts, pricing tiers). Still the source of truth consumed by the UI, pricing, and the dust layer.
2. model_constructors (front/lib/model_constructors/*) — provider-agnostic endpoint classes, one per (provider, model, region, provider-api). Each class mixes a shared provider base client with a per-model config mixin (input schema, context size, token pricing). This is where the real request/response shape and the narrowed input config live.
3. llms (dust layer) (front/lib/llms/*) — thin Dust-specific wrappers around the model_constructors classes that add Dust concerns (display name, byok, endpoint filters, and any caps — e.g. exposing 250k context on a model that natively supports 1M). Registered into DUST_STREAM_ENDPOINTS.

Endpoints are named and filed as: {provider}_{model}_{region}_{provider_api}.ts e.g. google_gemini_3_6_flash_global_agent_platform.ts. The class name is the PascalCase of the same, with numbers spelled out: GoogleGeminiThreeDotSixFlashGlobalAgentPlatformStream.

The fastest, most reliable way to add a model is to copy the most recent model in the same family across all layers and rename. Grep every reference to that model and mirror each one. This skill lists the reference points; the sibling model is your template.

Before you start: verify against official docs (MANDATORY)

You MUST confirm every value below against the provider's official documentation and leave a URL + date in a code comment next to it. Do not carry values over from memory.

• Specs (context window, max output tokens, vision, structured output):
  • OpenAI: https://platform.openai.com/docs/models
  • Anthropic: https://docs.anthropic.com/en/docs/about-claude/models/overview
  • Google: https://ai.google.dev/gemini-api/docs/models
  • Mistral: https://docs.mistral.ai/getting-started/models/models_overview/
• Pricing (input / output / cached input per 1M tokens):
  • OpenAI: https://openai.com/api/pricing/
  • Anthropic: https://www.anthropic.com/pricing#anthropic-api
  • Google: https://ai.google.dev/gemini-api/docs/pricing
  • Mistral: https://mistral.ai/technology/#pricing
• Host / region availability: verify which provider APIs and regions actually serve the model day-one. Mirror the sibling model's endpoints, but only register an endpoint whose region is actually available. Keep an unavailable-but-anticipated endpoint class defined and unregistered (see Gemini's EU agent-platform example) with a comment saying why.

WebSearch/WebFetch the docs first. If a value can't be confirmed, surface it — don't guess.

Reference points to mirror (grep the sibling model)

Pick the newest sibling (e.g. for "Gemini 3.6 Flash" the sibling is "Gemini 3.5 Flash") and grep -rln its id / const / class-name / model-id string. You will touch, roughly:

A. Model config + central registry

B. Pricing / tiers / reasoning (TYPE-ENFORCED over StaticModelIdType)

Adding the id to STATIC_MODEL_IDS makes these fail to compile until updated:

C. model_constructors — the endpoint classes (stream)

D. model_constructors — tests (TDD, see below)

E. llms — the dust layer (stream)

F. SDK + UI + marketing mirror

Batch endpoints (.../batch/...) are a curated subset — only add them if the model needs batch. They are NOT completeness-enforced. Set supportsBatchProcessing to the real capability regardless.

The TDD loop (steps to actually run)

The endpoint classes derive their behavior from a shared integration test harness. Let the live API tell you the input contract — never infer it from the sibling model. Sibling expectations are the single biggest source of wrong config: two models in the same family routinely differ on temperature, reasoning efforts, and forced tool use.

The config schema must ALWAYS mirror the API's real behavior as closely as possible. It describes what the provider accepts — not what Dust happens to send today, and not what would be convenient. If the API accepts a value, the schema accepts it; if the API rejects a value, the schema rejects it. Never narrow past the API because an upstream layer already strips the field (the dropTemperature / dropTemperatureWhenReasoning config parsers in lib/llms are a product policy and belong there, not in the endpoint schema), and never widen past it to avoid a union. Concretely: Anthropic reasoning models accept exactly temperature: 1, so the field is z.literal(1).optional().default(1) — not z.undefined(), even though the Dust layer drops it before the endpoint ever sees it.

When a divergence from the API is genuinely wanted (exposing a narrower effort set to control cost, say), it is a policy choice — write it as a comment stating that the API allows more and why Dust doesn't, so the next reader doesn't mistake it for a provider constraint.

Reasoning efforts must ALWAYS mirror the model's official documentation, not merely whatever the endpoint happens to accept. This is the one place where "what the API tolerates" is the wrong source of truth, because gateways are routinely looser than the models they serve:

• The Fireworks gateway validates reasoning_effort against low/medium/high/xhigh/max/none for every model it hosts, so a live run "passes" on efforts the model never defined.
• DeepSeek documents disabled/high/max and says low/medium are mapped to high and xhigh to max — accepting them would silently rewrite the caller's choice.
• Kimi K3 is documented low/high/max by Moonshot; medium works through Fireworks but is not a K3 eff

Content truncated.