my-llm-api

MyLLMAPI

A two-surface LLM gateway:

• Raw — complete / embed / models. You pick a self-hosted catalog model (run myapi llm models for the live list). Proprietary models are not callable here.
• Verbs — classify / extract / summarize / draft. You ask for a task done; the model is implementation detail.

Capabilities

<!-- llm:start -->

For workflow tasks — summarize, classify, extract, draft. The reply goes to stdout and the usage footer (tokens, cost, finish reason) to stderr, so myapi llm complete ... | jq works.

Don't use this as your own model. If you are an agent reading this, you already have a more capable model. Reach for the LLM verbs when scripting a recurring step where a small, cheap model is the right tool — not for one-shot reasoning you can do yourself.

Reach for raw complete when shape matters (you build the messages array and set max_tokens/temperature/stop); reach for a verb when you want a result and don't care which model produced it.

Raw complete request

{
  "model": "<model-id>",
  "messages": [
    { "role": "system", "content": "You are a terse editor." },
    { "role": "user",   "content": "Tighten this paragraph: ..." }
  ],
  "max_tokens": 200,
  "temperature": 0.2,
  "stop": ["\n\n"]
}

Roles: system | user | assistant. Multiple system messages collapse to one instruction. max_tokens, temperature, and stop are optional — per-model defaults apply.

Raw complete response

{
  "model": "<model-id>",
  "content": "...assistant reply...",
  "finish_reason": "stop",
  "usage": { "input_tokens": 42, "output_tokens": 87, "cost_cents": 0.029 }
}

finish_reason is one of stop (normal), length (hit max_tokens), filter (blocked).

If model isn't in the self-hosted catalog the server returns MODEL_NOT_IN_RAW_CATALOG — that's the signal to use a verb instead, not to retry with a different --model.

Raw embed

Embed text into a dense vector. --model is optional when the catalog serves exactly one embed model; else pass one from myapi llm models --kind embed. Returns EMBED_NOT_AVAILABLE when none is served.

Model catalog

• Chat models: id, kind: 'chat', context_window, input_cost_per_1m_cents, output_cost_per_1m_cents
• Embed models: id, kind: 'embed', dimensions, input_cost_per_1m_cents

The catalog is live — it reflects what the inference gateway actually serves, refreshed every 15 minutes. Always query models rather than hard-coding ids.

Verb requests + responses

Every verb takes an optional tier (fast | reasoning | cheap) — opaque routing hint, server picks the model. Response usage block is identical across verbs.

Verb	Request body	Response data
classify	{ input, labels: string[], multi?: boolean, tier? }	{ label } or { labels: string[] } (when multi)
extract	{ input, schema: <json-schema>, tier? }	{ data: <object conforming to schema> }
summarize	{ input, style?: 'brief'|'exec'|'bullet', tier? }	{ summary }
draft	{ input?, kind: string, context?: object, prompt?: string, tier? }	{ text }

Shared usage block on every verb:

{ "tier_used": "fast", "tokens_in": 65, "tokens_out": 37, "cost_cents": 0.005 }

The model/provider is never named in the verb response — the verb is the contract.

OpenAI-compatible drop-in

POST /llm/orgs/{org_id}/chat/completions (alias /v1/chat/completions) takes and returns the OpenAI shape — no envelope. Same catalog and pricing as complete. Use it when an existing OpenAI SDK or LangChain integration should point at MyAPI unchanged.

from openai import OpenAI
client = OpenAI(
    api_key="hq_live_…",
    base_url="https://api.myapihq.com/llm/orgs/<org_id>/v1",
)
r = client.chat.completions.create(model="<model-id>",
    messages=[{"role":"user","content":"Hi"}])

<!-- llm:end -->

Commands

<!-- generated:start -->

Command	What it does
myapi llm models [--kind chat|embed] [--json]	List the live model catalog with pricing (cents/1M)
myapi llm complete "<prompt>" [--model <id>] [--system "<s>"] [--max-tokens N] [--temperature 0..1] [--stop <csv>] [--file <path>] [--json]	Raw chat completion; reply to stdout, usage to stderr. Defaults to the first chat model in the catalog
myapi llm embed "<text>" [--model <id>] [--json]	Embed a string into a vector; --model optional when the catalog has one embed model
myapi llm classify "<input>" --labels <csv> [--multi] [--tier <t>] [--json]	Pick a label from a set
myapi llm extract "<input>" --schema <path|json> [--tier <t>] [--json]	Pull structured data conforming to a JSON Schema
myapi llm summarize "<input>" [--style brief|exec|bullet] [--tier <t>] [--json]	Summarize text
myapi llm draft --kind <what> [--prompt "<s>"] [--facts <json>] [--directives <json>] ["<src>"] [--tier <t>] [--json]	Draft an email / reply / message / …

<!-- generated:end -->

Pass - as the prompt/input to read from stdin. Pass --file <path> to read longer content from disk.

Examples

<!-- llm:start -->

# List the live catalog
myapi llm models
myapi llm models --kind chat --json | jq '.models[].id'

# Raw completion — picks the first chat model from the catalog
myapi llm complete "Summarize in 12 words: $(cat README.md)"

# Pin a specific model (ids come from `myapi llm models`)
myapi llm complete "Refactor this function: ..." \
  --model <model-id> \
  --system "You are a careful Go reviewer." \
  --max-tokens 600

# ── Verbs (recommended for workflow steps) ──────────────────────────────

myapi llm classify "I was charged twice — please refund." \
  --labels billing,technical,sales,spam

myapi llm extract "Acme Corp employs 250 people in Berlin." \
  --schema '{"type":"object","properties":{"company":{"type":"string"},"employees":{"type":"integer"}}}'

myapi llm summarize --file long-thread.txt --style bullet

myapi llm draft --kind email \
  --prompt "Friendly welcome, under 60 words." \
  --facts '{"recipient":"a new signup","product":"MyAPI"}'

# Classify + route an inbound webhook delivery
BODY=$(myapi webhook delivery <id> --json | jq -r '.body')
INTENT=$(printf '%s' "$BODY" | myapi llm classify - \
  --labels support,sales,spam --json | jq -r '.data.label')

<!-- llm:end -->

Notes

• draft --facts safety. Fact values are quoted into the prompt verbatim and sensitive-named keys (secret, api_key, password, …) are NOT redacted. Two guards: injection-defense strips instructions/system/prompt/override keys into meta.warnings; an output guardrail substring-scans fact values (≥4 chars) and lists hits in meta.guardrails.facts_in_output (signal, not redaction). Never put credentials, PII, or internal metadata in --facts — pass identifiers and reference them indirectly.
• 402 — INSUFFICIENT_FUNDS: top up or enable myapi billing auto-recharge. SPEND_CAP_EXCEEDED: raise your own ceiling with myapi billing spend-cap.
• Self-hosted raw, server-picked verbs. Raw runs on MyAPI's TPU; verbs route wherever the server picks.
• Cost + latency. usage.cost_cents is authoritative — no markup. Varies by tier: 200–600 ms to first token, 1–3 s end-to-end.
• Live catalog, no streaming, no BYOK. Don't hard-code ids — models is truth (CLI auto-picks if --model omitted). Full reply only.

--facts vs --directives on draft

--facts '<json>' is referent data, quoted as reference and never as instructions (recipient, dates, amounts). --directives '<json>' is writer controls only: tone, max_words, format, style. They are trusted differently.

myapi llm draft --kind email --prompt "the invoice is due" \
  --facts '{"to":"Ada"}' --directives '{"tone":"warm"}'

--context is the old name for --facts; accepted, deprecated upstream.

HTTP (from deployed code)

<!-- http:start --> <!-- generated by `npm run canonical-sync` — do not edit -->

base   https://api.myapihq.com
path   POST /llm/orgs/{org_id}/complete
auth   Authorization: Bearer <key>   (fn: env.__MYAPI_KEY · container: env.MYAPI_KEY)
reply  { "success": true, "data": …, "error": null, "meta": {…} }

• Per-slot host — do not assume one host serves every slot.
• Org id goes in the PATH — there is no X-Org-Id header.

<!-- http:end -->