ai-debug-harness

AI Debug Harness — Skill

This skill teaches an AI coding agent (or a human) how to drive the self-hosted debug harness for the VoiceAgent stack. The full design lives under blogs/ai-coding-agent-debug-system.

Trigger conditions

Load and follow this skill when ANY of these are true:

• The user reports the desktop app "doesn't connect" or "no audio."
• The renderer logs BUNDLE PT=... collisions or NotAllowedError.
• A test or run produces NDJSON under .debug/logs/<runId>/ and you need to reason about it.
• The user asks to "reproduce", "bisect", or "diagnose" anything in the voice-chat path.

Loop invariant

You MUST follow this loop until the user task is fully resolved:

1. Preflight — prefer recent successful doctor evidence when available. Use .debug/state/doctor-latest.json if it is fresh, successful, covers the requested mode, and its input hash still matches. Otherwise run pnpm debug:doctor for full checks or let debug:e2e/debug:repro request compact quick preflight. If any check has ok: false, fix the misconfiguration and re-run. Do NOT proceed until doctor passes.
2. Reproduce — pnpm debug:e2e (or pnpm debug:repro -- <scenario>). This produces a runId and a directory .debug/logs/<runId>/.
3. Read the run — open .debug/runs/<runId>/report.json. Note the failedAt step and the matchedCookbookIds.
4. Match against the cookbook — for each id in matchedCookbookIds, read the entry in docs/debug/cookbook.md and apply its fix_pattern.
5. If no match — read the FIRST line with level: "error" in the per-source NDJSON files (preference: doctor → main → renderer → agent → api-gateway → livekit → whisper-api → fish-speech-api). Diagnose, fix, then append a new cookbook entry capturing the cause. Use appendNewEntry() from scripts/debug/cookbook.ts or hand-author the frontmatter block.
6. Re-run the harness. If green, set the matching entry's verified_in_run to the green run's id.
7. Stop when harness.run.summary shows status: "passed" AND zero un-whitelisted error lines.

Reading NDJSON

Every line is { ts, level, source, runId, event, correlationId?, payload? }. Use jq filters:

jq -c 'select(.level == "error")' .debug/logs/<runId>/*.ndjson
jq -c 'select(.event == "renderer.console.error")' .debug/logs/<runId>/renderer.ndjson

The schema reference lives at blogs/ai-coding-agent-debug-system/ndjson-schema.md.

Adding a cookbook entry

Format spec: blogs/ai-coding-agent-debug-system/cookbook-format.md.

Required frontmatter fields: id, symptom_jq, sources, likely_cause, fix_pattern, verified_in_run (start as null, set to the run id once a clean run reproduces+resolves the issue).

Commands

Command	Purpose
pnpm debug:doctor	Full synchronous preflight (Docker, LiveKit, GPU, ports, env)
pnpm debug:e2e	Full Playwright Electron E2E with NDJSON capture
pnpm debug:repro	Re-run a specific scenario by name and write report.json
pnpm debug:bisect	git bisect run pnpm debug:e2e between two refs
pnpm debug:clean	Prune .debug/ directories older than 7 days

Hard rules

• NEVER edit a renderer file to call Node APIs directly. All disk I/O goes through window.electronAPI (contextBridge).
• NEVER set expected: true on a cookbook entry to silence a real error.
• NEVER commit anything under .debug/ (it is gitignored).
• ALWAYS run pnpm typecheck && pnpm test before declaring a fix done.
• Keep user-facing debug summaries compact: report the run id, failed step, first failing source, unmatched error count, and cookbook ids instead of pasting full NDJSON logs unless the user asks.