using-openrouter

Using OpenRouter (for this CV app)

OpenRouter gives one prepaid balance + one OpenAI-compatible key for ~every model. You call it with the official openai SDK by swapping base_url. The whole reason this skill exists is that one default behavior will silently break us, plus a few model-specific quirks. Internalize the four rules below and use the bundled helper.

scripts/or_client.py encodes all of this — import it, don't re-derive it. references/openrouter-reference.md has the exhaustive, sourced facts + per-provider tables. Read it when a detail here isn't enough or when slugs/providers need re-verifying.

The four rules that matter

1. Always pin require_parameters for any JSON call. By default OpenRouter may route your request to a provider that doesn't support response_format and will silently ignore your schema — you get HTTP 200 with free-form prose, no error. The fix is extra_body={"provider": {"require_parameters": True}}, which routes only to providers that honor every parameter you sent. This is non-negotiable for deepseek-v4-pro (its cheapest/default endpoint, DeepSeek's own, does not enforce strict schema).

2. gpt-5.1 rejects a custom temperature. Same quirk as on OpenAI direct (the GPT-5 family). Send temperature only to Qwen/DeepSeek; for gpt-5.1 rely on schema + seed.

3. Turn reasoning OFF for JSON/judging calls. qwen3.7-max and deepseek-v4-pro are reasoning-capable. Thinking tokens are billed as output, add latency/nondeterminism, and some providers drop or leak them under json_schema. Send extra_body={"reasoning": {"enabled": False, "exclude": True}}. qwen3-235b-a22b-2507 is the non-thinking Instruct variant — cleanest for deterministic temp-0 judging.

4. Non-streaming + Response Healing for reliability. Use stream=False so OpenRouter's free plugins:[{"id":"response-healing"}] can repair malformed JSON syntax (Qwen/DeepSeek emit invalid JSON a few % of the time; healing takes Qwen3-235B ~88% → ~99.98% valid). Healing fixes syntax, not schema adherence — always still validate the parsed dict and fail closed.

Verified model slugs (2026-06-17 — re-verify before long-term reliance)

Slugs/providers drift. Confirm with strict_providers(slug) (in the helper) or GET /api/v1/models/{slug}/endpoints before trusting a provider list.

The canonical call (use the helper)

from scripts.or_client import or_client, chat_json, MODELS, STRICT_PROVIDERS

client = or_client()  # OpenAI SDK pointed at OpenRouter, key from .env

# Strict CV generation (reasoning off, provider pinned, healing on, cost returned):
cv, meta = chat_json(
    client, MODELS["deepseek-v4-pro"],
    messages=[{"role": "system", "content": system_prompt},
              {"role": "user", "content": jobs_xml}],
    schema=cv_schema, schema_name="tailored_cv",
    pin_providers=STRICT_PROVIDERS["deepseek/deepseek-v4-pro"],  # belt-and-suspenders
)
print(meta["served_by"], meta["cost_usd"])

# Deterministic temp-0 judge (non-thinking model, temp honored):
verdict, meta = chat_json(
    client, MODELS["qwen3-235b"],
    messages=judge_msgs, schema=judge_schema, schema_name="coverage",
    temperature=0, seed=7,
)

chat_json already: sets require_parameters, disables reasoning, enables healing, omits temperature for gpt-5.*, strips stray ```json fences, json.loads (raises on bad output — fail closed), and pulls cost/tokens from resp.model_dump()["usage"].

Strict-schema requirements (our build_cv_schema already complies)

Strict mode = every object has additionalProperties: false, every property is in required, optional fields are nullable unions ({"type": ["array", "null"]}), and constraint keywords (pattern, min/maxLength, format, default, …) are stripped/ignored. server/cv_schema.py:build_cv_schema already follows this exactly — reuse it, don't rewrite.

Cost, credits, errors (details in the reference)

• Cost is auto-returned: resp.model_dump()["usage"]["cost"] (USD; the typed .usage.cost may be absent). Ground truth: GET /api/v1/generation?id=<id> (retry on 404 — written async).
• Check balance before a batch: check_key() in the helper (GET /api/v1/key).
• Retry 408/429/502/503/504; never retry 402 (no credits) or 403 (guardrail). require_parameters + too many filters → 503 (no eligible provider) — loosen a filter.

When to read the reference

references/openrouter-reference.md — per-provider strict-SO tables for each model, the full reasoning/billing model, prompt-caching specifics, privacy/data controls, the complete error table, and every source URL. Go there before changing provider pins or debugging a routing/ schema/temperature surprise.