siro-pilot
Benchmarks AI models on fixed tasks to provide cost-per-promotion evidence.
Install
mkdir -p .claude/skills/siro-pilot && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/12854" && unzip -o skill.zip -d .claude/skills/siro-pilot && rm skill.zipInstalls to .claude/skills/siro-pilot
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Run the bounded siro operational pilot from Codex and interpret its cost-per-promotion report. Use when the user wants to compare Tier 0 / cheap-frontier / strong-frontier arms on the fixed task list, estimate cost per promotion, or get a continue/revise/stop recommendation before any scale-up.Key capabilities
- →Initialize the siro operational pilot
- →Run the siro pilot with specified arms
- →Generate a cost-per-promotion report
- →Interpret pilot outcomes per arm (accepted, mixed, failed)
- →Report estimated spend and cost per accepted promotion
- →Provide a continue/revise/stop recommendation
How it works
The skill initializes, runs, and reports on a budget-capped comparison of three arms (Tier 0, cheap frontier, strong frontier) on a fixed task list, producing a cost-per-promotion report.
Inputs & outputs
When to use siro-pilot
- →Compare model costs
- →Test frontier model performance
- →Get promotion recommendations
- →Audit AI task spending
About this skill
Run the siro operational pilot
A fixed, budget-capped comparison of three arms (Tier 0 local, cheap frontier, strong frontier) on an immutable task list, ending in a Markdown cost-per-promotion report with a continue/revise/stop recommendation. The pilot approves no scale-up by itself: it only produces evidence for a human decision.
The three steps
Run in order:
uv run siro pilot-init
uv run siro pilot-run
uv run siro pilot-report
pilot-initis reproducible and writes the plan under the pilot root; the task list and per-arm configs are fixed. Do not edit them to flatter a result.pilot-runruns the required arms (Tier 0 + cheap frontier). Add--include-conditionalto also run the strong-frontier follow-up; run a single arm with--arm <name>.pilot-reportrenders from archived research attempts plus model-call ledgers toreports/; pass--provider-reconciliation "<note/url>"if you have dashboard figures.
The frontier arms make real provider calls and cost money. Before running them, confirm the user wants to spend and that any needed budget headroom is already approved. The Tier 0 arm is free and safe to run unprompted.
Interpret the report
Read the rendered report and relay, concisely:
- Outcomes per arm: accepted/promoted vs mixed/escalated vs failed.
- Cost: estimated spend, cost per accepted promotion, cost per family.
- Integrity rates: hidden-test / reproducibility / safety rates.
- Flags: any budget breach or missing evidence the report calls out.
- Recommendation: the report's continue/revise/stop verdict. Present it as the report's recommendation, and make clear that acting on it is a separate human-gated governance step.
Surface the file the report was written to so the user can open it.
When not to use it
- →When the user needs to scale up operations directly
- →When the user needs to modify the task list or per-arm configs
Limitations
- →The pilot approves no scale-up by itself
- →The task list and per-arm configs are fixed
- →The skill requires explicit confirmation for frontier arms due to cost
How it compares
This skill automates a bounded, comparative pilot for AI models, providing objective cost and performance metrics for decision-making, which is more rigorous than ad-hoc model testing.
Compared to similar skills
siro-pilot side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| siro-pilot (this skill) | 0 | 1mo | Review | Advanced |
| quant-analyst | 103 | 2mo | No flags | Advanced |
| llava | 7 | 8mo | Review | Advanced |
| cocoindex | 6 | 9mo | Review | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
You might also like
quant-analyst
zenobi-us
Expert quantitative analyst specializing in financial modeling, algorithmic trading, and risk analytics. Masters statistical methods, derivatives pricing, and high-frequency trading with focus on mathematical rigor, performance optimization, and profitable strategy development.
llava
zechenzhangAGI
Large Language and Vision Assistant. Enables visual instruction tuning and image-based conversations. Combines CLIP vision encoder with Vicuna/LLaMA language models. Supports multi-turn image chat, visual question answering, and instruction following. Use for vision-language chatbots or image understanding tasks. Best for conversational image analysis.
cocoindex
cocoindex-io
Comprehensive toolkit for developing with the CocoIndex library. Use when users need to create data transformation pipelines (flows), write custom functions, or operate flows via CLI or API. Covers building ETL workflows for AI data processing, including embedding documents into vector databases, building knowledge graphs, creating search indexes, or processing data streams with incremental updates.
ai-multimodal
mrgoonie
Process and generate multimedia content using Google Gemini API. Capabilities include analyze audio files (transcription with timestamps, summarization, speech understanding, music/sound analysis up to 9.5 hours), understand images (captioning, object detection, OCR, visual Q&A, segmentation), process videos (scene detection, Q&A, temporal analysis, YouTube URLs, up to 6 hours), extract from documents (PDF tables, forms, charts, diagrams, multi-page), generate images (text-to-image, editing, composition, refinement). Use when working with audio/video files, analyzing images or screenshots, processing PDF documents, extracting structured data from media, creating images from text prompts, or implementing multimodal AI features. Supports multiple models (Gemini 2.5/2.0) with context windows up to 2M tokens.
rag-implementation
wshobson
Build Retrieval-Augmented Generation (RAG) systems for LLM applications with vector databases and semantic search. Use when implementing knowledge-grounded AI, building document Q&A systems, or integrating LLMs with external knowledge bases.
umap-learn
K-Dense-AI
UMAP dimensionality reduction. Fast nonlinear manifold learning for 2D/3D visualization, clustering preprocessing (HDBSCAN), supervised/parametric UMAP, for high-dimensional data.