Creates synthetic data pairs based on task specifications to power benchmarks.

Install

mkdir -p .claude/skills/synthesize-data && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/14179" && unzip -o skill.zip -d .claude/skills/synthesize-data && rm skill.zip

Installs to .claude/skills/synthesize-data

Activation

This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.

Generate synthetic benchmark examples from a task spec. Invoked from setup-data when there is no data, or directly when the user asks to generate examples. Produces a JSONL file in .data/<benchmark-name>/.
205 chars✓ has a “when” trigger
Intermediate

Key capabilities

  • →Read benchmark specifications
  • →Generate synthetic text and expected output pairs
  • →Validate generated results against field schema
  • →Save generated data to a JSONL file
  • →Update benchmark spec to point to generated data
  • →Warn about small datasets for statistical reliability

How it works

The skill reads a benchmark specification, builds a generation prompt, calls a generator model, validates results, and saves them to a JSONL file.

Inputs & outputs

You give it
A benchmark specification with a valid `task:` section
You get back
A `.jsonl` file containing synthetic `{text, expected}` pairs and an updated benchmark spec

When to use synthesize-data

  • →Generate test data for benchmarks
  • →Create synthetic evaluation examples
  • →Validate benchmark task specs

About synthesize-data

Reads benchmark requirements and generates sample input-output pairs. Validates output against the specification, ensuring robust testing data.

Generate synthetic benchmark examples from a task spec. Invoked from setup-data when there is no data, or directly when the user asks to generate examples. Produces a JSONL file in .data/<benchmark-name>/.

When not to use it

  • →When `task.input.type` is `image`, `document`, or `pdf`
  • →When real images are required for the benchmark

Limitations

  • →The skill cannot generate images, documents, or PDFs.
  • →The skill relies on an internal generator model that is not exposed to the user.

How it compares

This skill automates the creation of synthetic benchmark data based on a specification, providing structured and validated examples, which differs from manually crafting test data.

Compared to similar skills

synthesize-data side by side with the closest alternatives in the catalog.

SkillInstallsUpdatedSafetyDifficulty
synthesize-data (this skill)03moReviewIntermediate
data-engineering139moReviewAdvanced
crawl4ai2110moReviewIntermediate
data-cleaning-pipeline137moReviewIntermediate

Try saying

Example prompts that trigger this skill in your AI assistant.

Search skills

Search the agent skills registry