verification-quality-assurance

Verification & Quality Assurance Skill

What This Skill Does

This skill provides a comprehensive verification and quality assurance system that ensures code quality and correctness through:

• Truth Scoring: Real-time reliability metrics (0.0-1.0 scale) for code, agents, and tasks
• Verification Checks: Automated code correctness, security, and best practices validation
• Automatic Rollback: Instant reversion of changes that fail verification (default threshold: 0.95)
• Quality Metrics: Statistical analysis with trends, confidence intervals, and improvement tracking
• CI/CD Integration: Export capabilities for continuous integration pipelines
• Real-time Monitoring: Live dashboards and watch modes for ongoing verification

Shipped vs. aspirational. The concrete, in-CI verification stack — the 6 regression-guard jobs + the witness manifest + the tool-discoverability audit — is real and runs on every push. The truth-scoring / auto-rollback / WebSocket-dashboard surface described later in this doc is partly shipped (ruflo verify runs the witness checks) and partly design — treat the "CI Guards" section below as the authoritative current state.

CI Guards — what's actually shipped (current state)

Ruflo's regression protection is three layers, all gated before publish. Authoritative reference: verification/README.md.

Run the guards locally

# Tool-description discoverability audit (ADR-112)
node scripts/audit-tool-descriptions.mjs                       # fails if any baseline count rises
node scripts/audit-tool-descriptions.mjs --update-baseline     # lock the new floor after a fix lands

# Behavioral smokes (each builds what it needs; safe to run individually)
node plugins/ruflo-core/scripts/test-hooks.mjs "node $PWD/v3/@claude-flow/cli/bin/cli.js"
node plugins/ruflo-core/scripts/test-mcp-protocol.mjs
node plugins/ruflo-core/scripts/test-memory-import.mjs
node plugins/ruflo-core/scripts/test-mcp-roundtrips.mjs        # #1889 paired-tool round-trip
node plugins/ruflo-core/scripts/test-cli-no-crash.mjs          # #1863 unhandled-exception class
node plugins/ruflo-core/scripts/test-consensus-transport.mjs   # ADR-095 G2 consensus transport

# Witness manifest — regenerate + verify
node scripts/regen-witness.mjs
node plugins/ruflo-core/scripts/witness/verify.mjs --manifest verification/macos/manifest.md.json

# Temporal history
node plugins/ruflo-core/scripts/witness/history.mjs --history verification/macos/history.jsonl summary
node plugins/ruflo-core/scripts/witness/history.mjs --history verification/macos/history.jsonl regressions

Adding a new guard

1. Behavioral smoke → write plugins/ruflo-core/scripts/test-<name>.mjs. Pattern: static dist-scan first (fast, always completes), behavioral probe second with an internal timeout + a process-level watchdog so CI never hangs. Add a step to the relevant job in v3-ci.yml.
2. Static gate with a baseline → write scripts/audit-<name>.mjs that scans, counts violations, and fails if the count exceeds a monotone-decreasing baseline in verification/<name>-baseline.json. Support --update-baseline. Add a CI job; wire it into witness-verify needs[] if it should gate publish.
3. Documented-fix marker → append { id, desc, file, marker } to verification/witness-fixes.json, run node scripts/regen-witness.mjs. The marker must be a substring the fix specifically creates (not a generic pattern like 'function').

Prerequisites

• Ruflo installed (npx ruflo@alpha)
• Git repository (for rollback features)
• Node.js 18+ (for dashboard features)
• @noble/ed25519 (for the witness verifier — a single runtime dep, npm i @noble/ed25519)

Quick Start

# View current truth scores
npx ruflo@alpha truth

# Run verification check
npx ruflo@alpha verify check

# Verify specific file with custom threshold
npx ruflo@alpha verify check --file src$app.js --threshold 0.98

# Rollback last failed verification
npx ruflo@alpha verify rollback --last-good

Complete Guide

Truth Scoring System

View Truth Metrics

Display comprehensive quality and reliability metrics for your codebase and agent tasks.

Basic Usage:

# View current truth scores (default: table format)
npx ruflo@alpha truth

# View scores for specific time period
npx ruflo@alpha truth --period 7d

# View scores for specific agent
npx ruflo@alpha truth --agent coder --period 24h

# Find files$tasks below threshold
npx ruflo@alpha truth --threshold 0.8

Output Formats:

# Table format (default)
npx ruflo@alpha truth --format table

# JSON for programmatic access
npx ruflo@alpha truth --format json

# CSV for spreadsheet analysis
npx ruflo@alpha truth --format csv

# HTML report with visualizations
npx ruflo@alpha truth --format html --export report.html

Real-time Monitoring:

# Watch mode with live updates
npx ruflo@alpha truth --watch

# Export metrics automatically
npx ruflo@alpha truth --export .claude-flow$metrics$truth-$(date +%Y%m%d).json

Truth Score Dashboard

Example dashboard output:

📊 Truth Metrics Dashboard
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Overall Truth Score: 0.947 ✅
Trend: ↗️ +2.3% (7d)

Top Performers:
  verification-agent   0.982 ⭐
  code-analyzer       0.971 ⭐
  test-generator      0.958 ✅

Needs Attention:
  refactor-agent      0.821 ⚠️
  docs-generator      0.794 ⚠️

Recent Tasks:
  task-456  0.991 ✅  "Implement auth"
  task-455  0.967 ✅  "Add tests"
  task-454  0.743 ❌  "Refactor API"

Metrics Explained

Truth Scores (0.0-1.0):

• 1.0-0.95: Excellent ⭐ (production-ready)
• 0.94-0.85: Good ✅ (acceptable quality)
• 0.84-0.75: Warning ⚠️ (needs attention)
• <0.75: Critical ❌ (requires immediate action)

Trend Indicators:

• ↗️ Improving (positive trend)
• → Stable (consistent performance)
• ↘️ Declining (quality regression detected)

Statistics:

• Mean Score: Average truth score across all measurements
• Median Score: Middle value (less affected by outliers)
• Standard Deviation: Consistency of scores (lower = more consistent)
• Confidence Interval: Statistical reliability of measurements

Verification Checks

Run Verification

Execute comprehensive verification checks on code, tasks, or agent outputs.

File Verification:

# Verify single file
npx ruflo@alpha verify check --file src$app.js

# Verify directory recursively
npx ruflo@alpha verify check --directory src/

# Verify with auto-fix enabled
npx ruflo@alpha verify check --file src$utils.js --auto-fix

# Verify current working directory
npx ruflo@alpha verify check

Task Verification:

# Verify specific task output
npx ruflo@alpha verify check --task task-123

# Verify with custom threshold
npx ruflo@alpha verify check --task task-456 --threshold 0.99

# Verbose output for debugging
npx ruflo@alpha verify check --task task-789 --verbose

Batch Verification:

# Verify multiple files in parallel
npx ruflo@alpha verify batch --files "*.js" --parallel

# Verify with pattern matching
npx ruflo@alpha verify batch --pattern "src/**/*.ts"

# Integration test suite
npx ruflo@alpha verify integration --test-suite full

Verification Criteria

The verification system evaluates:

1. Code Correctness

   • Syntax validation
   • Type checking (TypeScript)
   • Logic flow analysis
   • Error handling completeness

2. Best Practices

   • Code style adherence
   • SOLID principles
   • Design patterns usage
   • Modularity and reusability

3. Security

   • Vulnerability scanning
   • Secret detection
   • Input validation
   • Authentication$authorization checks

4. Performance

   • Algorithmic complexity
   • Memory usage patterns
   • Database query optimization
   • Bundle size impact

5. Documentation

   • JSDoc/TypeDoc completeness
   • README accuracy
   • API documentation
   • Code comments quality

JSON Output for CI/CD

# Get structured JSON output
npx ruflo@alpha verify check --json > verification.json

# Example JSON structure:
{
  "overallScore": 0.947,
  "passed": true,
  "threshold": 0.95,
  "checks": [
    {
      "name": "code-correctness",
      "score": 0.98,
      "passed": true
    },
    {
      "name": "security",
      "score": 0.91,
      "passed": false,
      "issues": [...]
    }
  ]
}

Automatic Rollback

Rollback Failed Changes

Automatically revert changes that fail verification checks.

Basic Rollback:

# Rollback to last known good state
npx ruflo@alpha verify rollback --last-good

# Rollback to sp

---

*Content truncated.*