Ensure engine changes are correct by running parity tests against real dataset benchmarks.

Install

mkdir -p .claude/skills/verify-freeeve && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/18987" && unzip -o skill.zip -d .claude/skills/verify-freeeve && rm skill.zip

Installs to .claude/skills/verify-freeeve

Activation

This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.

Verify gochickpeas engine/gql changes end-to-end -- parity gate against real LDBC data plus a public-API sample drive.
118 charsno explicit “when” trigger
Intermediate

Key capabilities

  • Run full-pipeline parity gates against LDBC data
  • Check plan shapes for quality regressions
  • Verify engine changes end-to-end
  • Test public API with sample drives
  • Regenerate golden plans for intentional planner changes

How it works

The skill runs `gqlbench` with specified manifests and verification flags to compare actual output against reference hashes and check plan shapes. It also drives a sample through the public export using a scratchpad module.

Inputs & outputs

You give it
gochickpeas engine/gql changes
You get back
Verification results showing MATCH, DIFF, SKIP counts and plan-shape status

When to use verify

  • Verifying engine changes
  • Running parity benchmarks
  • Checking plan quality
  • Full suite testing

About this skill

Verifying gochickpeas changes

The surface is the library's public API. Two handles, use both for nontrivial engine/gql changes:

1. Parity gate (real data, full pipeline)

go run ./cmd/gqlbench -manifest ~/rustychickpeas-ldbc/viz/data/gql_variants.tsv \
  -verify-only -cached-parity \
  -plans-golden cmd/gqlbench/testdata/plans_golden.txt

Expect 89/89 MATCH, 0 DIFF, 0 SKIP, plus plan-shape golden: 89 queries unchanged. This drives parse -> plan -> exec over SF1/FinBench SF10 exports with pinned row hashes. Loads ~26M rels; takes a few minutes. Never emit (-verify-only) from a dirty tree -- the append-only bench-out protocol stamps engineCommit.

  • -cached-parity also checks the auto-parameterized PlanCache path against the same reference hashes (catches literal-vs-cached-plan divergence).

  • -plans-golden guards plan QUALITY, which row parity cannot see: a planner change that stays correct still moves the plan, and drift here fails the run. For an INTENTIONAL planner change, review the drift, then regenerate the golden with -plans-golden-capture and commit it in the same change. Do a planner change WITHOUT this and a regression that stays row-correct lands invisibly.

  • Run heavy invocations under the shared-box lock -- and not just the gate: full builds, full test runs, fuzz runs, and benchmarks are exactly the multi-core work that lands on top of an ldbc sweep mid-measurement.

    taskman lock run -ttl 20m -wait 30m -reason "<what>" local-cpu -- go build ./...
    taskman lock run -ttl 20m -wait 30m -reason "<what>" local-cpu -- go test ./...
    

    Incremental builds, go vet, and editor-driven builds are fine unlocked. A non-zero exit from lock run means another session holds the box: wait or do unrelated work -- never run the job anyway.

2. Sample drive through the public export

Scratchpad module with a replace directive resolves the local repo:

module verifydrive
require github.com/freeeve/gochickpeas v0.0.0
replace github.com/freeeve/gochickpeas => /Users/efreeman/gochickpeas

Build a small graph with chickpeas.NewBuilder (AddNode/SetProp/AddRel/ SetRelPropAt, then Finalize("name")), run queries with gql.Run, and inspect plans with gql.Explain (e.g. grep the [mono ...] marker on VarExpand lines to see whether a pushdown fired). Probe near-miss phrasings and malformed input; errors should be clean plan/bind errors.

Gotchas:

  • Timing on this machine is very noisy; alloc counts (gqlbench profiles output) are the most trustworthy A/B signal. For timing A/Bs, run the whole comparison inside ONE taskman lock run local-cpu session in ABBA order (new-old-old-new): interleaving alone does not cancel a load trend, only ABBA does. A run whose TIMING is the product must also pass -max-load 2 and must not publish on a non-zero exit.
  • gqlbench must run from the repo root (HeadStamp shells to git); point -out/-plans-out/-profiles-out at the scratchpad.
  • 45s of go test ./gql -fuzz FuzzQuery -fuzztime 45s is cheap insurance after recognizer/planner changes.

When not to use it

  • When the tree is dirty, do not emit with -verify-only
  • When another session holds the box, do not run the job anyway

Limitations

  • Timing measurements are noisy on the machine
  • gqlbench must run from the repo root
  • Cannot emit with -verify-only from a dirty tree

How it compares

This skill automates end-to-end verification with real data and plan quality checks, unlike manual testing that might miss regressions or require separate tools.

Compared to similar skills

verify side by side with the closest alternatives in the catalog.

SkillInstallsUpdatedSafetyDifficulty
verify (this skill)013dReviewIntermediate
run-tests14moReviewIntermediate
advanced-lokstra-validate-consistency05moReviewAdvanced
go-dev-guidelines148moNo flagsIntermediate

Try saying

Example prompts that trigger this skill in your AI assistant.

You might also like

run-tests

pgschema

Run pgschema automated tests (go test) to validate diff logic, plan generation, and dump functionality using test fixtures

13

advanced-lokstra-validate-consistency

primadi

Validate application consistency - circular dependencies, schema validation, config checks, annotation validation, and service registration. Use after all code is implemented to identify issues before deployment.

00

go-dev-guidelines

jumppad-labs

This skill should be used when writing, refactoring, or testing Go code. It provides idiomatic Go development patterns, TDD-based workflows, project structure conventions, and testing best practices using testify/require and mockery. Activate this skill when creating new Go features, services, packages, tests, or when setting up new Go projects.

1495

code-formatting

openshift

MANDATORY: When writing Go tests, you MUST use 'When...it should...' format for ALL test names. When writing any Go code, you MUST remind user to run 'make lint-fix' and 'make verify'. These are non-negotiable HyperShift requirements.

112

tidb-test-guidelines

pingcap

Decide where to place TiDB tests and how to write them (basic structure, naming, testdata usage). Use when asked about test locations, writing conventions, shard_count limits, casetest categorization, or when reviewing test changes in code review.

210

go-table-driven-tests

Xe

Write Go table-driven tests following Go community best practices and this repository's conventions. Use when writing or refactoring Go tests, especially when you notice repeated test patterns or copy-pasted test code.

15

Search skills

Search the agent skills registry