kiln-add-model

Add a New AI Model to Kiln

Integrating a new model into libs/core/kiln_ai/adapters/ml_model_list.py requires:

1. ModelName enum – add an enum member
2. built_in_models list – add a KilnModel(...) entry with providers
3. ModelFamily enum – only if the vendor is brand-new

After code changes, run paid integration tests, then draft a Discord post.

Global Rules

These apply throughout the entire workflow.

• Slug verification: NEVER guess or infer model slugs from naming patterns. Every model_id must come from an authoritative source (LiteLLM catalog, official docs, API reference, or changelog). If you can't verify a slug, tell the user and ask them to provide it.
• Date awareness: These models are often released very recently. Web search for current info before assuming you know the details.

Phase 1 – Model Discovery (only when asked to find new/missing models)

If the user asks you to find new models, do NOT just web search "new AI models this week" — that only surfaces major releases. Instead, systematically check each family against both the LiteLLM catalog and models.dev, then union the results. Both are attempts to catalog available models and each has gaps the other fills.

1. Read the ModelFamily and ModelName enums to know what we already have.

2. Query both catalogs for each family (run in parallel where possible):

   LiteLLM catalog — filters out mirror providers to avoid duplicates:

   curl -s 'https://api.litellm.ai/model_catalog?model=SEARCH_TERM&mode=chat&page_size=500' -H 'accept: application/json' | jq '[.data[] | select(.provider != "openrouter" and .provider != "bedrock" and .provider != "bedrock_converse" and .provider != "vertex_ai-anthropic_models" and .provider != "azure") | .id] | unique | .[]'
   

   models.dev — search all model IDs across all providers:

   curl -s https://models.dev/api.json | jq '[to_entries[].value.models // {} | keys[]] | .[]' | grep -i "SEARCH_TERM"
   

   For details on a specific provider+model: curl -s https://models.dev/api.json | jq '.["PROVIDER"].models["MODEL_ID"]'

3. Search terms (one query per term): claude, gpt, o1, o3, o4 (OpenAI reasoning), gemini, llama, deepseek, qwen, qwq, mistral, grok, kimi, glm, minimax, hunyuan, ernie, phi, gemma, seed, step, pangu

4. Union and cross-reference results from both catalogs against ModelName. A model found in either source counts as available. Focus on direct-provider entries (not OpenRouter/Bedrock/Azure mirrors). Skip pure coding models (e.g. codestral, deepseek-coder, qwen-coder).

5. Run targeted web searches per family to catch very fresh releases not yet in either catalog:

   • "[family] new model [current year]"
   • "[family] release [current month] [current year]"

6. Present findings as a summary. Let the user decide which to add.

Phase 1B – Lagging-Provider Backfill Check (every run)

Some providers — Fireworks AI, Together AI, SiliconFlow — expose new models on their own endpoints 1–2 weeks before those entries surface in models.dev / LiteLLM. Relying only on those two catalogs will both under-populate the provider list for the model you're adding now and miss the window to backfill recently-added models whose provider support has since grown.

Run this check on every invocation of the skill, regardless of whether you're in discovery mode or adding a specific model.

1. Pull the 10 most recently added models from the top of built_in_models in ml_model_list.py (newest are at the top), or from git:

   git log --follow -p -- libs/core/kiln_ai/adapters/ml_model_list.py | grep -E "^\+\s+name=ModelName\." | head -20
   

2. For the model you're adding (if any) AND each of those 10 models, cross-check Fireworks, Together, and SiliconFlow directly using the endpoints in the Lagging Providers Reference. Do NOT trust models.dev / LiteLLM as the final word for these three providers.

3. If a lagging provider now supports a recently-added model that isn't yet in its KilnModel entry, flag it to the user and propose either bundling the provider addition into the current change or opening a separate PR. Do not silently add it.

Phase 2 – Gather Context

1. Read the predecessor model in ml_model_list.py (e.g. for Opus 4.6 → read Opus 4.5). You inherit most parameters from it.

2. Query the LiteLLM catalog for the new model. This is the primary slug source since Kiln uses LiteLLM. See the Slug Lookup Reference for query syntax and all verified sources.

3. Get the OpenRouter slug via:

   • curl -s https://openrouter.ai/api/v1/models | jq '.data[].id' | grep -i "SEARCH_TERM"
   • Fallback: WebSearch for openrouter [model name] model id

4. Get the direct-provider slug (Anthropic, OpenAI, Google, etc.). Use the LiteLLM catalog first, then official docs. See the Slug Lookup Reference for provider-specific URLs.

5. Identify quirks — check the Provider Quirks Reference for the relevant provider, and web search for any new quirks:

   • Structured output mode (JSON schema vs function calling)?
   • Reasoning model (needs reasoning_capable, parsers, OpenRouter options)?
   • Vision/multimodal support? Which MIME types?
   • Provider-specific flags (temp_top_p_exclusive, etc.)?
   • Rate limit concerns (max_parallel_requests)?

6. Determine thinking levels — does the model support configurable reasoning effort? See Thinking Levels Reference for the full lookup chain. Key quick checks:

   • Check the vendor model page (e.g. OpenAI model pages say "Reasoning.effort supports: X, Y, Z")
   • Check OpenRouter supported_parameters — if reasoning is absent, skip thinking levels
   • R1-style thinking models (DeepSeek, Qwen thinking variants) do NOT get thinking level dicts

Phase 3 – Code Changes

All changes go in libs/core/kiln_ai/adapters/ml_model_list.py.

3a. ModelName enum

• snake_case: claude_opus_4_6 = "claude_opus_4_6"
• Place before predecessor (newer first within group)
• Follow existing grouping (all claude together, all gpt together, etc.)

3b. KilnModel entry in built_in_models

• Place before predecessor entry (newer = higher in list)
• Copy predecessor's structure and modify: name, friendly_name, model_id per provider, flags
• friendly_name must follow the existing naming pattern of sibling models in the same family. Check the predecessor. For example, Claude Sonnets use "Claude {version} Sonnet" (e.g. "Claude 4.5 Sonnet"), not "Claude Sonnet {version}". Do NOT use the vendor's marketing name if it differs from Kiln's established convention.

Provider model_id formats:

Every single model_id must be verified from an authoritative source. No exceptions.

Setting flags — use catalog data + predecessor as dual signals:

The LiteLLM catalog and models.dev responses include capability flags (supports_vision, supports_function_calling, supports_reasoning, etc.). Use these as the primary signal for what to enable on the new model:

• If the catalog says supports_vision: true → enable supports_vision, multimodal_capable, and vision MIME types (see 2c)
• If the catalog says supports_function_calling: true → use StructuredOutputMode.json_schema (or function_calling depending on provider norms — check predecessor)
• If the catalog says supports_reasoning: true → the model can reason, but do NOT reflexively set reasoning_capable=True — default to reasoning_capable=False (see Reasoning Capable Default). Still add available_thinking_levels if it supports effort levels, and check parser/formatter flags.

Then cross-check against the predecessor. The predecessor tells you how Kiln configures a similar model (which structured_output_mode, which provider-specific flags, etc.). The catalog tells you what the model can do. Use both:

• Catalog says the model supports vision but predecessor doesn't have it? Enable it — this is a new capability.
• Predecessor has temp_top_p_exclusive but nothing in the catalog mentions it? Keep it — it's a provider quirk the catalog doesn't track.
• Catalog and predecessor disagree on something? Trust the catalog for capabilities, trust the predecessor for Kiln-specific configuration patterns.

Common flags:

• structured_output_mode – how the model handles JSON output
• suggested_for_evals / suggested_for_data_gen – see zero-sum rule below
• multimodal_capable / supports_vision / supports_doc_extraction – see multimodal rules below
• reasoning_capable – for thinking/reasoning models. Default new models to reasoning_capable=False unless the model always emits its reasoning (see Reasoning Capable Default)
• temp_top_p_exclusive – Anthropic models that can't have both temp and top_p
• `parser

Content truncated.