workflow-test-fix-cycle

Workflow Test-Fix Cycle

End-to-end test-fix workflow pipeline: generate test sessions with progressive layers (L0-L3), AI code validation, and task generation (Phase 1), then execute iterative fix cycles with adaptive strategy engine until pass rate >= 95% (Phase 2).

Architecture Overview

┌────────────────────────────────────────────────────────────────────────────┐
│  Workflow Test-Fix Cycle Orchestrator (SKILL.md)                            │
│  → Full pipeline: Test generation + Iterative execution                     │
│  → Phase dispatch: Read phase docs, execute, pass context                   │
└───────────────┬────────────────────────────────────────────────────────────┘
                │
   ┌────────────┴────────────────────────┐
   ↓                                     ↓
┌─────────────────────────┐   ┌─────────────────────────────┐
│ Phase 1: Test-Fix Gen   │   │ Phase 2: Test-Cycle Execute  │
│ phases/01-test-fix-gen  │   │ phases/02-test-cycle-execute │
│ 5 sub-phases:           │   │ 3 stages:                    │
│ ① Create Session        │   │ ① Discovery                  │
│ ② Gather Context        │   │ ② Main Loop (iterate)        │
│ ③ Test Analysis (Gemini)│   │ ③ Completion                 │
│ ④ Generate Tasks        │   │                              │
│ ⑤ Summary               │   │ Agents (via spawn_agent):    │
│                         │   │ @cli-planning-agent           │
│ Agents (via spawn_agent)│   │ @test-fix-agent               │
│ @test-context-search    │   │                              │
│ @context-search         │   │ Strategy: conservative →      │
│ @cli-execution          │   │ aggressive → surgical          │
│ @action-planning        │   │                              │
└────────┬────────────────┘   └────────────┬──────────────────┘
         ↓                                 ↓
   IMPL-001..002.json              Pass Rate >= 95%
   TEST_ANALYSIS_RESULTS.md        Auto-complete session

Task Pipeline:
┌──────────────┐    ┌─────────────────┐    ┌─────────────────┐    ┌──────────────┐
│  IMPL-001    │───→│  IMPL-001.3     │───→│  IMPL-001.5     │───→│  IMPL-002    │
│  Test Gen    │    │  Code Validate  │    │  Quality Gate   │    │  Test & Fix  │
│  L1-L3       │    │  L0 + AI Issues │    │  Coverage 80%+  │    │  Max 10 iter │
│@code-developer│   │ @test-fix-agent │    │ @test-fix-agent │    │@test-fix-agent│
└──────────────┘    └─────────────────┘    └─────────────────┘    └──────────────┘
                                                                        │
                                                              Fix Loop: │
                                                    ┌──────────────────┘
                                                    ↓
                                              ┌──────────┐
                                              │ @cli-plan│───→ IMPL-fix-N.json
                                              │  agent   │
                                              ├──────────┤
                                              │@test-fix │───→ Apply & re-test
                                              │  agent   │
                                              └──────────┘

Key Design Principles

1. Two-Phase Pipeline: Generation (Phase 1) creates session + tasks, Execution (Phase 2) runs iterative fix cycles
2. Pure Orchestrator: Dispatch to phase docs, parse outputs, pass context between phases
3. Phase 1 Auto-Continue: Sub-phases within Phase 1 run autonomously
4. Subagent Lifecycle: Explicit lifecycle management with spawn_agent → wait_agent → close_agent
5. Progressive Test Layers: L0 (Static) → L1 (Unit) → L2 (Integration) → L3 (E2E)
6. AI Code Issue Detection: Validates against common AI-generated code problems
7. Intelligent Strategy Engine: conservative → aggressive → surgical based on iteration context
8. CLI Fallback Chain: Gemini → Qwen → Codex for analysis resilience
9. Progressive Testing: Affected tests during iterations, full suite for final validation
10. Role Path Loading: Subagent roles loaded via path reference in MANDATORY FIRST STEPS

Auto Mode

Phase 1 generates test session and tasks. Phase 2 executes iterative fix cycles until pass rate >= 95% or max iterations reached. Between Phase 1 and Phase 2, you MUST stop and wait for user confirmation before proceeding to execution. Phase 2 runs autonomously once approved.

Subagent API Reference

spawn_agent

Create a new subagent with task assignment.

const agentId = spawn_agent({
  agent_type: "{agent_type}",
  message: `
## TASK ASSIGNMENT

### MANDATORY FIRST STEPS (Agent Execute)
1. Run: `ccw spec load --category "planning execution"`

## TASK CONTEXT
${taskContext}

## DELIVERABLES
${deliverables}
`
})

wait_agent

Get results from subagent (only way to retrieve results).

const result = wait_agent({
  timeout_ms: 1800000  // 30 minutes
})

if (result.timed_out) {
  followup_task({ target: agentId, message: "STATUS_CHECK: Report current progress, findings so far, and estimated remaining work." })
  const status = wait_agent({ timeout_ms: 180000 })  // 3 min
  if (status.timed_out) {
    followup_task({ target: agentId, message: "FINALIZE: Output all current findings immediately. Time limit reached.", interrupt: true })
    const forced = wait_agent({ timeout_ms: 180000 })  // 3 min
    if (forced.timed_out) {
      close_agent({ target: agentId })
    }
  }
}

followup_task

Assign new work to active subagent (for clarification or follow-up).

followup_task({
  target: agentId,
  message: `
## CLARIFICATION ANSWERS
${answers}

## NEXT STEP
Continue with plan generation.
`
})

close_agent

Clean up subagent resources (irreversible).

close_agent({ target: agentId })

Usage

workflow-test-fix-cycle <input> [options]

# Input (Phase 1 - Test Generation)
source-session-id    WFS-* session ID (Session Mode - test validation for completed implementation)
feature description  Text description of what to test (Prompt Mode)
/path/to/file.md     Path to requirements file (Prompt Mode)

# Options (Phase 2 - Cycle Execution)
--max-iterations=N   Custom iteration limit (default: 10)

# Examples
workflow-test-fix-cycle WFS-user-auth-v2                                              # Session Mode
workflow-test-fix-cycle "Test the user authentication API endpoints in src/auth/api.ts" # Prompt Mode - text
workflow-test-fix-cycle ./docs/api-requirements.md                                     # Prompt Mode - file
workflow-test-fix-cycle "Test user registration" --max-iterations=15                    # With custom iterations

# Resume (Phase 2 only - session already created)
workflow-test-fix-cycle --resume-session="WFS-test-user-auth"                          # Resume interrupted session

Quality Gate: Test pass rate >= 95% (criticality-aware) or 100% Max Iterations: 10 (default, adjustable) CLI Tools: Gemini → Qwen → Codex (fallback chain)

Test Strategy Overview

Progressive Test Layers (L0-L3):

Key Features:

• AI Code Issue Detection - Validates against common AI-generated code problems (hallucinated imports, placeholder code, mock leakage, etc.)
• Project Type Detection - Applies appropriate test templates (React, Node API, CLI, Library, etc.)
• Quality Gates - IMPL-001.3 (code validation) and IMPL-001.5 (test quality) ensure high standards

Detailed specifications: See the test-task-generate workflow tool for complete L0-L3 requirements and quality thresholds.

Execution Flow

Input → Detect Mode (session | prompt | resume)
  │
  ├─ resume mode → Skip to Phase 2
  │
  └─ session/prompt mode → Phase 1
       │
Phase 1: Test-Fix Generation (phases/01-test-fix-gen.md)
  ├─ Sub-phase 1.1: Create Test Session → testSessionId
  ├─ Sub-phase 1.2: Gather Test Context (spawn_agent) → contextPath
  ├─ Sub-phase 1.3: Test Generation Analysis (spawn_agent → Gemini) → TEST_ANALYSIS_RESULTS.md
  ├─ Sub-phase 1.4: Generate Test Tasks (spawn_agent) → IMPL-*.json, IMPL_PLAN.md, TODO_LIST.md
  └─ Sub-phase 1.5: Phase 1 Summary
       │
  ⛔ MANDATORY CONFIRMATION GATE
  │   Present plan summary → request_user_input → User approves/cancels
  │   NEVER auto-proceed to Phase 2
       │
Phase 2: Test-Cycle Execution (phases/02-test-cycle-execute.md)
  ├─ Discovery: Load session, tasks, iteration state
  ├─ Main Loop (for each task):
  │   ├─ Execute → Test → Calculate pass_rate
  │   ├─ 100% → SUCCESS: Next task
  │   ├─ 95-99% + low criticality → PARTIAL SUCCESS: Approve
  │   └─ <95% → Fix Loop:
  │       ├─ Select strategy: conservative/aggressive/surgical
  │       ├─ spawn_agent(@cli-planning-agent) → IMPL-fix-N.json
  │       ├─ spawn_agent(@test-fix-agent) → Apply fix & re-test
  │       └─ Re-test → Back to decision
  └─ Completion: Final validation → Summary → Sync session state → Auto-complete session

Core Rules

1. Start Immediately: First action is functions.update_plan initialization
2. No Preliminary Analysis: Do not read files before Phase 1
3. Parse Every Output: Extract data from each phase/sub-phase for the next
4. Within-Phase Auto-Continue: Sub-phases within a phase run automatically; Phase 2 iterations run automatically once started
5. Phase Loading: Read phase doc on-demand (phases/01-*.md, phases/02-*.md)
6. Task Attachment Model: Sub-tasks ATTACH → execute → COLLAPSE
7. MANDATORY CONFIRMATION GATE: After Phase 1 completes, you MUST stop and present the generated plan to the user. Wait

Content truncated.