dust-llm

Adding Support for a New LLM Model

This skill guides you through adding a newly released LLM to the model_constructors + llms stack (the endpoint-class router). It replaces the legacy lib/api/llm/clients/* router, which no longer exists.

Adding a model is usually only half the task: the model it supersedes has to be retired in the same PR, and a model the provider has switched off needs its agents repointed. See Deprecating or removing an old model.

Mental model

A model reaches production through three stacked layers. Add the new model to each:

1. Model config (front/types/assistant/models/*) — the legacy ModelConfigurationType describing the model (context, vision, reasoning efforts, pricing tiers). Still the source of truth consumed by the UI, pricing, and the dust layer.
2. model_constructors (front/lib/model_constructors/*) — provider-agnostic endpoint classes, one per (provider, model, region, provider-api). Each class mixes a shared provider base client with a per-model config mixin (input schema, context size, token pricing). This is where the real request/response shape and the narrowed input config live.
3. llms (dust layer) (front/lib/llms/*) — thin Dust-specific wrappers around the model_constructors classes that add Dust concerns (display name, byok, endpoint filters, and any caps — e.g. exposing 250k context on a model that natively supports 1M). Registered into DUST_STREAM_ENDPOINTS.

Endpoints are named and filed as: {provider}_{model}_{region}_{provider_api}.ts e.g. google_gemini_3_6_flash_global_agent_platform.ts. The class name is the PascalCase of the same, with numbers spelled out: GoogleGeminiThreeDotSixFlashGlobalAgentPlatformStream.

The fastest, most reliable way to add a model is to copy the most recent model in the same family across all layers and rename. Grep every reference to that model and mirror each one. This skill lists the reference points; the sibling model is your template.

Before you start: verify against official docs (MANDATORY)

You MUST confirm every value below against the provider's official documentation and leave a URL + date in a code comment next to it. Do not carry values over from memory.

• Specs (context window, max output tokens, vision, structured output):
  • OpenAI: https://platform.openai.com/docs/models
  • Anthropic: https://docs.anthropic.com/en/docs/about-claude/models/overview
  • Google: https://ai.google.dev/gemini-api/docs/models
  • Mistral: https://docs.mistral.ai/getting-started/models/models_overview/
• Pricing (input / output / cached input per 1M tokens):
  • OpenAI: https://openai.com/api/pricing/
  • Anthropic: https://www.anthropic.com/pricing#anthropic-api
  • Google: https://ai.google.dev/gemini-api/docs/pricing
  • Mistral: https://mistral.ai/technology/#pricing
• Host / region availability: verify which provider APIs and regions actually serve the model day-one. Mirror the sibling model's endpoints, but only register an endpoint whose region is actually available. Keep an unavailable-but-anticipated endpoint class defined and unregistered (see Gemini's EU agent-platform example) with a comment saying why.

WebSearch/WebFetch the docs first. If a value can't be confirmed, surface it — don't guess.

Reference points to mirror (grep the sibling model)

Pick the newest sibling (e.g. for "Gemini 3.6 Flash" the sibling is "Gemini 3.5 Flash") and grep -rln its id / const / class-name / model-id string. You will touch, roughly:

A. Model config + central registry

B. Pricing / tiers / reasoning (TYPE-ENFORCED over StaticModelIdType)

Adding the id to STATIC_MODEL_IDS makes these fail to compile until updated:

And one that is not compile-forced, so nothing turns red if you skip it:

EU pricing is a second, silent list. Any endpoint with region = EUROPE bills through inferenceRegion: "eu" (inferenceRegionForEndpointRegion in front/lib/api/llm/transitionLLM.ts), and computeTokensCostForUsageInMicroUsd then looks the model up in EU_MODEL_PRICING — falling back to the global rate when it is absent. EU_UPLIFT_MODEL_IDS is satisfies readonly StaticModelIdType[], which validates the ids present but does not force completeness, so a missing entry undercharges EU traffic forever with nothing failing.

The uplift is per provider and per endpoint, not per model — compare the two endpoint classes' tokenPricing rather than assuming. Regional agent-platform (Vertex) endpoints charge 10% over global for both Anthropic and Google, so a new Gemini registered on eu/agent-platform belongs in the list just as much as a Claude does. OpenAI uplifts only the models whose pricing page lists a data-residency premium (gpt-5.4/5.5/5.6/6 yes, gpt-5/5.1/5.2 no). Mistral's own models are EU-only with no global sibling, so nothing to add for them.

EU_MODEL_PRICING derives every field by multiplying the global entry by EU_PRICING_MULTIPLIER, so it is only correct when the EU endpoint is a flat 1.1× of global. A non-uniform regional price needs an explicit entry in EU_HOST_MODEL_PRICING, not the multiplier — always the case when the EU host differs from the global one (GLM-5.3: Fireworks global, Mistral EU).

Third-party model on a lab's own host (e.g. GLM-5.3 on Mistral): set lab on the endpoint (a host serving several labs leaves it off its base client), map the host's model name with modelToHostModel, and check PROVIDER_ID_TO_HOST in front/lib/api/llm/index.ts. Routing matches lab ∈ whitelisted labs OR host ∈ whitelisted hosts, so an endpoint whose lab and host are both unmapped is silently unreachable.

Gating is inherited, and lives in two unlinked places. A new version of a gated model stays gated — being newer is not a reason to release it. Copy the predecessor's availableIfOneOf / unavailableIfOneOf onto the new X_MODEL_CONFIG (gates the picker, via isModelAvailable) and declare the same flag on every endpoint you add (gates the router, via isEndpointAvailable):

static readonly endpointFilter = {
  featureFlags: { contains: "fireworks_new_model_feature" as const },
};

Half-gating fails silently either way: hidden but reachable, or pickable but unroutable — and resolveModel swaps in a fallback model instead of erroring. Releasing a gated family is a separate, deliberate change.

An eu/agent-platform endpoint takes EU_AGENT_PLATFORM_ENDPOINT_FILTER, never {}. Dust-managed EU hosting is sold to credit-priced plans and to workspaces carrying use_vertex_for_supported_models; that rule lives once, in front/lib/llms/utils/endpoint_filters.ts, and every *_eu_agent_platform.ts dust wrapper assigns it verbatim:

static readonly endpointFilter = EU_AGENT_PLATFORM_ENDPOINT_FILTER;

Copy the constant, not a sibling's inline literal — a hand-written copy is how six Gemini Flash EU endpoints ended up on {}, routing legacy workspaces to EU hosting nobody had promised them while the picker showed no EU flag. The client mirrors the same constant in useRunsOnRegionalHosting (front/hooks/useRunsOnRegionalHosting.ts) to decide whether to show a workspace its hosting region, so an endpoint that opts out makes that indicator lie.

A model-specific gate composes with it rather than replacing it — { and: [FILTER, { featureFlags: … }] }. This is about region = EUROPE on host = AGENT_PLATFORM only: provider-hosted EU endpoints (*_eu_openai_responses, *_eu_mistral) run on the provider's own EU infrastructure, are available to everyone, and keep endpointFilter = {}.

C. model_constructors — the endpoint classes (stream)

Content truncated.