devtu-optimize-skills

Optimizing ToolUniverse Skills

Best practices for high-quality research skills with evidence grading and source attribution.

Tool Quality Standards

1. Error messages must be actionable — tell the user what went wrong AND what to do
2. Schema must match API reality — run python3 -m tooluniverse.cli run <Tool> '<json>' to verify
3. Coverage transparency — state what data is NOT included
4. Input validation before API calls — don't silently send invalid values
5. Cross-tool routing — name the correct tool when query is out-of-scope
6. No silent parameter dropping — if a parameter is ignored, say so

Core Principles (13 Patterns)

Full details: references/optimization-patterns.md

Optimized Skill Workflow

Phase -1: Tool Verification (check params)
Phase  0: Foundation Data (aggregator query)
Phase  1: Disambiguation (IDs, collisions, baseline)
Phase  2: Specialized Queries (fill gaps)
Phase  3: Report Synthesis (evidence-graded narrative)

Testing Standards

Full details: references/testing-standards.md

Critical rule: NEVER write skill docs without testing all tool calls first.

• 30+ tests per skill, 100% pass rate
• All tests use real data (no placeholders)
• Phase + integration + edge case tests
• SOAP tools (IMGT, SAbDab, TheraSAbDab) need operation parameter
• Distinguish transient errors (retry) from real bugs (fix)
• API docs are often wrong — always verify with actual calls

Pattern 14: Reasoning Frameworks Over Tool Catalogs (CRITICAL)

Skills that just list tools ("call A, then B, then C") score 3-5/10 in usefulness tests. Skills that explain HOW to interpret and combine data score 7-9/10. Every skill MUST include:

14a. Interpretation Tables

Map raw API data to biological/clinical meaning. Don't just retrieve — explain.

14b. Synthesis Phases

Every multi-phase skill needs a final phase that answers "so what?" — not just collecting data:

• "What changed and why does it matter?"
• "Is this cause or consequence?"
• "What's the actionable recommendation?"

14c. Honest Limitations

If a tool API can't deliver what the skill promises, say so explicitly. Don't describe aspirational capabilities. Example: "DepMap_get_gene_dependencies returns gene metadata only, NOT per-cell-line CRISPR scores."

Pattern 15: Computational Procedures When Tools Can't Help

Some scientific analyses require computation, not just API queries. When no tool exists for a capability, embed a Python code procedure directly in the skill using packages available in ToolUniverse (pandas, scipy, numpy, statsmodels, biopython, networkx).

When to use computational procedures:

Template for computational procedures in skills:

**Computational procedure: [Name]**
[When to use this: explain the gap it fills]

\`\`\`python
# [What this computes]
# Requires: [packages] (included in ToolUniverse dependencies)
import pandas as pd
from scipy.stats import mannwhitneyu

# Input: [describe expected input format]
# Output: [describe output]
# [Full working code with example data]
\`\`\`

[Interpretation guidance for the output]

Key rules for computational procedures:

1. Only use packages in ToolUniverse dependencies (pyproject.toml): pandas, scipy, numpy, networkx, requests, biopython (optional extra)
2. Include example data so the procedure is immediately testable
3. Explain the output — a code block without interpretation is useless
4. Note when external data download is needed (e.g., DepMap CSV from depmap.org)

Pattern 15b: Download-and-Process for Datasets Without REST APIs

Many critical scientific datasets have NO REST API but provide bulk download files. Skills should include concrete download-and-process instructions when this is the only path to essential data.

Template for download-and-process procedures:

**Step 1: Download data files**
- URL: [exact download page URL]
- Files needed: [filename] (~[size]) — [what it contains]
- Registration: [required/not required]
- Update frequency: [quarterly/annually/etc.]

**Step 2: Process with Python**
[Working code with pandas/scipy that loads the CSV and produces the analysis]

**Step 3: Interpret results**
[Table mapping output values to biological/clinical meaning]

**When files are not available**: [Fallback strategy using API tools]

Known download-only datasets that skills reference:

Critical rule: Always include a fallback for when the download is unavailable (user may not have registration, file may be too large, etc.). The fallback should use available API tools even if they provide less complete data.

Common Anti-Patterns

Quick Fixes for User Complaints

Skill Template

---
name: [domain]-research
description: [What + when triggers]
---

# [Domain] Research

## Workflow
Phase -1: Tool Verification → Phase 0: Foundation → Phase 1: Disambiguate
→ Phase 2: Search → Phase 3: Report

## Phase -1: Tool Verification
[Parameter corrections table]

## Phase 0: Foundation Data
[Aggregator query]

## Phase 1: Disambiguation
[IDs, collisions, baseline]

## Phase 2: Specialized Queries
[Query strategy, fallbacks]

## Phase 3: Report Synthesis
[Evidence grading, mandatory sections]

## Output Files
- [topic]_report.md, [topic]_bibliography.json

## Quantified Minimums
[Numbers per section]

## Completeness Checklist
[Required sections with checkbo

---

*Content truncated.*