verify
Ensure engine changes are correct by running parity tests against real dataset benchmarks.
Install
mkdir -p .claude/skills/verify-freeeve && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/18987" && unzip -o skill.zip -d .claude/skills/verify-freeeve && rm skill.zipInstalls to .claude/skills/verify-freeeve
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Verify gochickpeas engine/gql changes end-to-end -- parity gate against real LDBC data plus a public-API sample drive.Key capabilities
- →Run full-pipeline parity gates against LDBC data
- →Check plan shapes for quality regressions
- →Verify engine changes end-to-end
- →Test public API with sample drives
- →Regenerate golden plans for intentional planner changes
How it works
The skill runs `gqlbench` with specified manifests and verification flags to compare actual output against reference hashes and check plan shapes. It also drives a sample through the public export using a scratchpad module.
Inputs & outputs
When to use verify
- →Verifying engine changes
- →Running parity benchmarks
- →Checking plan quality
- →Full suite testing
About this skill
Verifying gochickpeas changes
The surface is the library's public API. Two handles, use both for nontrivial engine/gql changes:
1. Parity gate (real data, full pipeline)
go run ./cmd/gqlbench -manifest ~/rustychickpeas-ldbc/viz/data/gql_variants.tsv \
-verify-only -cached-parity \
-plans-golden cmd/gqlbench/testdata/plans_golden.txt
Expect 89/89 MATCH, 0 DIFF, 0 SKIP, plus plan-shape golden: 89 queries unchanged. This drives parse -> plan -> exec over SF1/FinBench SF10 exports
with pinned row hashes. Loads ~26M rels; takes a few minutes. Never emit
(-verify-only) from a dirty tree -- the append-only bench-out protocol stamps
engineCommit.
-
-cached-parityalso checks the auto-parameterized PlanCache path against the same reference hashes (catches literal-vs-cached-plan divergence). -
-plans-goldenguards plan QUALITY, which row parity cannot see: a planner change that stays correct still moves the plan, and drift here fails the run. For an INTENTIONAL planner change, review the drift, then regenerate the golden with-plans-golden-captureand commit it in the same change. Do a planner change WITHOUT this and a regression that stays row-correct lands invisibly. -
Run heavy invocations under the shared-box lock -- and not just the gate: full builds, full test runs, fuzz runs, and benchmarks are exactly the multi-core work that lands on top of an ldbc sweep mid-measurement.
taskman lock run -ttl 20m -wait 30m -reason "<what>" local-cpu -- go build ./... taskman lock run -ttl 20m -wait 30m -reason "<what>" local-cpu -- go test ./...Incremental builds,
go vet, and editor-driven builds are fine unlocked. A non-zero exit fromlock runmeans another session holds the box: wait or do unrelated work -- never run the job anyway.
2. Sample drive through the public export
Scratchpad module with a replace directive resolves the local repo:
module verifydrive
require github.com/freeeve/gochickpeas v0.0.0
replace github.com/freeeve/gochickpeas => /Users/efreeman/gochickpeas
Build a small graph with chickpeas.NewBuilder (AddNode/SetProp/AddRel/
SetRelPropAt, then Finalize("name")), run queries with gql.Run, and
inspect plans with gql.Explain (e.g. grep the [mono ...] marker on
VarExpand lines to see whether a pushdown fired). Probe near-miss
phrasings and malformed input; errors should be clean plan/bind errors.
Gotchas:
- Timing on this machine is very noisy; alloc counts (gqlbench profiles
output) are the most trustworthy A/B signal. For timing A/Bs, run the
whole comparison inside ONE
taskman lock run local-cpusession in ABBA order (new-old-old-new): interleaving alone does not cancel a load trend, only ABBA does. A run whose TIMING is the product must also pass-max-load 2and must not publish on a non-zero exit. gqlbenchmust run from the repo root (HeadStamp shells to git); point -out/-plans-out/-profiles-out at the scratchpad.- 45s of
go test ./gql -fuzz FuzzQuery -fuzztime 45sis cheap insurance after recognizer/planner changes.
When not to use it
- →When the tree is dirty, do not emit with -verify-only
- →When another session holds the box, do not run the job anyway
Limitations
- →Timing measurements are noisy on the machine
- →gqlbench must run from the repo root
- →Cannot emit with -verify-only from a dirty tree
How it compares
This skill automates end-to-end verification with real data and plan quality checks, unlike manual testing that might miss regressions or require separate tools.
Compared to similar skills
verify side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| verify (this skill) | 0 | 13d | Review | Intermediate |
| run-tests | 1 | 4mo | Review | Intermediate |
| advanced-lokstra-validate-consistency | 0 | 5mo | Review | Advanced |
| go-dev-guidelines | 14 | 8mo | No flags | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
You might also like
run-tests
pgschema
Run pgschema automated tests (go test) to validate diff logic, plan generation, and dump functionality using test fixtures
advanced-lokstra-validate-consistency
primadi
Validate application consistency - circular dependencies, schema validation, config checks, annotation validation, and service registration. Use after all code is implemented to identify issues before deployment.
go-dev-guidelines
jumppad-labs
This skill should be used when writing, refactoring, or testing Go code. It provides idiomatic Go development patterns, TDD-based workflows, project structure conventions, and testing best practices using testify/require and mockery. Activate this skill when creating new Go features, services, packages, tests, or when setting up new Go projects.
code-formatting
openshift
MANDATORY: When writing Go tests, you MUST use 'When...it should...' format for ALL test names. When writing any Go code, you MUST remind user to run 'make lint-fix' and 'make verify'. These are non-negotiable HyperShift requirements.
tidb-test-guidelines
pingcap
Decide where to place TiDB tests and how to write them (basic structure, naming, testdata usage). Use when asked about test locations, writing conventions, shard_count limits, casetest categorization, or when reviewing test changes in code review.
go-table-driven-tests
Xe
Write Go table-driven tests following Go community best practices and this repository's conventions. Use when writing or refactoring Go tests, especially when you notice repeated test patterns or copy-pasted test code.