RE

regression-testing

Orchestrates regression test pipelines and provides actionable release-readiness insights.

Install

mkdir -p .claude/skills/regression-testing && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/18122" && unzip -o skill.zip -d .claude/skills/regression-testing && rm skill.zip

Installs to .claude/skills/regression-testing

Activation

This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.

Execute regression test suites via CI/CD, analyze results, classify failures, and produce GO/NO-GO release decisions. Use when running regression, smoke, or sanity suites through GitHub Actions, monitoring workflow runs, downloading Allure or Playwright artifacts, classifying failures (REGRESSION vs FLAKY vs KNOWN vs ENVIRONMENT vs NEW TEST), computing pass-rate and trend metrics, deciding release readiness, generating executive quality reports, or creating regression issues. Triggers on: run regression, trigger test workflow, analyze test results, quality report, GO/NO-GO decision, release readiness, flaky tests, Allure report, smoke suite, pass rate, nightly test failure, stage 6. Do NOT use for writing new regression tests (that belongs to test-automation) or for manual fix verification (that belongs to sprint-testing).
834 chars✓ has a “when” triggerlonger than Claude Code's old 250-char listing cap (fine on current versions)
Advanced

Key capabilities

  • Trigger CI suite runs through GitHub Actions
  • Monitor CI workflow runs to completion
  • Classify test failures into categories like REGRESSION or FLAKY
  • Compute pass-rate and trend metrics for test results
  • Generate executive quality reports for stakeholders
  • Create regression issues in Jira based on analysis

How it works

The skill orchestrates the release-readiness pipeline by triggering a CI suite, monitoring its completion, classifying failures, and scoring against release criteria. It then emits a verdict and a stakeholder report.

Inputs & outputs

You give it
GitHub workflow files (.github/workflows/*.yml), master test plan (.context/master-test-plan.md), Playwright config (playwright.config.ts), previous Allure repo
You get back
GO/NO-GO release decision, executive quality reports, regression issues in Jira

When to use regression-testing

  • Trigger smoke test suites
  • Analyze nightly test failures
  • Generate quality reports for stakeholders
  • Classify flaky vs. regression tests

About this skill

Forbidden invocations

NEVER invoke /sdd-* skills from this workflow. SDD is an optional user-installed ceremony; this skill ships self-contained and does not chain SDD under any condition. If you need to refactor KATA, fixtures, cli/, scripts/, or api/schemas/ pipeline, exit this skill first and invoke /framework-development — which itself runs Plan → Code → Verify → Archive natively (no SDD required).

This boundary is mechanical, not advisory: scripts/lint-skills.ts rejects any /sdd- mention outside this section. See: .claude/skills/agentic-qa-core/references/skill-composition-strategy.md §4 (governs users who manually install SDD).

Regression Testing — Execute, Analyze, Decide

Orchestrates the full release-readiness pipeline: trigger a CI suite, monitor it to completion, classify failures, score against release criteria, and emit a GO / CAUTION / NO-GO verdict plus a stakeholder report.

Three phases, always in this order: Execute → Analyze → Report. Do not skip analysis and jump to a report. Do not guess classification without reading failure logs.


Inputs

  • .github/workflows/*.yml — workflow files for regression / smoke / sanity suites; defines triggers, inputs, and artifact uploads.
  • .context/master-test-plan.md — regression Epic key + expected pass-rate SLOs per suite.
  • playwright.config.ts — reporter config, retry policy, project matrix; needed to interpret retry counts and shard splits.
  • Previous run's Allure report (artifact URL or local download under ./analysis/previous/) — baseline for trend computation.
  • kata-manifest.json — registry of tests and ATCs available; used to cross-reference failed test IDs.
  • .agents/jira-required.yaml — Jira refs (project key, work types, transitions) for filing regression issues.
  • agentic-qa-core/references/defect-management-doctrine.mdcanonical authority for classifying (Bug/Defect/Improvement), the mandatory field matrix, QA-Assignee ownership, and the QA process epic when a confirmed regression is filed in Jira (Phase 3). Read BEFORE filing any defect.

Subagent Dispatch Strategy

Orchestration & Session contracts: this skill follows ./orchestration-doctrine.md (mandatory subagent dispatch — main thread is command center) AND ./session-management.md (Phase 0 resume check, plan-first persistence at .session/<skill-slug>/<scope>/, archive on completion). Phase 0 (resume check) and Phase 1 (plan write) are NOT optional. The orchestrator also applies the per-stage Definition-of-Done gates in ./stage-gates.md: verify a stage's DoD BEFORE recording its progress checkpoint and advancing.

This skill is per-run scope: <scope> = <env>-<YYYY-MM-DD> (e.g. staging-2026-05-20). Session state lives at .session/regression-testing/<scope>/{plan.md, progress.md} per agentic-qa-core/references/session-management.md §3 + §9. The single highest-value resume case: if the Monitor subagent dies while watching a long CI run but RUN_ID was captured in plan.md, Phase 0 re-attaches via gh run view <RUN_ID> instead of re-triggering CI (saves 20–60 min of wall-clock).

This skill is compliant with the doctrine in CLAUDE.md §"Orchestration Mode (Subagent Strategy)" and the session contract in .claude/skills/agentic-qa-core/references/session-management.md. Every dispatch follows the 6-component briefing format defined in .claude/skills/agentic-qa-core/references/briefing-template.md, and the pattern selected per stage matches the decision guide in .claude/skills/agentic-qa-core/references/dispatch-patterns.md. The two CI-bound stages (long-running watch, multi-artifact download) and the high-volume failure classification step are the hotspots — everything else stays inline because the dispatch overhead is not justified.

StagePatternSubagent role
Trigger workflow (gh workflow run)Singleinline — no dispatch needed (one shell call)
Wait/monitor gh run watchBackgroundone Monitor subagent runs the watch; main thread continues with prep work; subagent notifies on exit
Download 3 artifacts (allure / evidence / playwright)Parallel3 simultaneous subagents, one per artifact; cap = 3 (no rate-limit risk)
Classify failures (chunks of ~10 tests each)ParallelN subagents based on failure volume; cap = 10 to avoid context dilution
Compute metrics (pass-rate, trends)Singleinline — needs aggregated state, low cost
Generate executive reportSingleinline — final synthesis, decisions live here
GO / CAUTION / NO-GO verdictSingleinline — main thread owns release decisions
  • Error protocol: On any subagent failure: STOP, report full context to user, present retry / skip / abort options. Do NOT auto-fix. See .claude/skills/agentic-qa-core/references/orchestration-doctrine.md.

Readiness Preflight Gate (MANDATORY — runs before Phase 0)

Full doctrine: agentic-qa-core/references/preflight-gate.md. Runs FIRST, before the resume check and any gh workflow run. Two laws: (1) args-as-answers — the suite (regression/smoke/sanity), env, and any grep/test_file are provided args; ask only the gaps. (2) probe, don't assume. Surface gaps + REDs as ONE AskUserQuestion checklist; self-fix with approval + explanation; STOP on any blocking RED. This generalizes the Phase 1 §Preflight (gh auth) to a full readiness check pulled to t=0. Generic baseline (env resolution, secret/restart handling, the two laws, output contract) is inherited from the reference §3.1 — not repeated here. Below is only this skill's specific capability delta (note: test-user creds, MCPs and browsers live inside the CI runner, not the orchestrator).

CapabilityNeedWhy here
GitHub CLI authenticatedREQUIREDEvery stage drives CI via gh (gh auth status, gh workflow run, gh run watch, gh run download). Not authed → user runs gh auth login (suggest the ! prefix); do not proceed.
Workflow files presentREQUIRED.github/workflows/ must hold the regression/smoke/sanity workflow for the chosen suite, with the inputs this skill passes.
GitHub Actions Secrets/VariablesREQUIREDThe runner authenticates with env-prefixed creds (secrets.<ENV>_USER_EMAIL / _PASSWORD) + XRAY_* / ATLASSIAN_* as Repository/Environment Secrets — the suite 401s mid-run without them. gh secret list (add --env <env> for environment scope) shows them; missing → gh secret set <NAME> from .env. /adapt-framework only emits a manual list today, so this is the most common silent gap.
Allure 3 localREQUIREDbunx allure resolves (devDep, no global install); allurerc.mjs present for bun allure:agent markdown triage.
Active envREQUIREDThe suite runs against <<ACTIVE_ENV>> (default {{DEFAULT_ENV}}). Confirm it is the intended target before a 20–60 min run.
[TMS_TOOL] (result sync)OPTIONALOnly when .agents/project.yaml testing.tms_cli is set — Stage 3 pushes run status. jira-xray → /xray-cli + XRAY_*.
[ISSUE_TRACKER_TOOL] (file regression issues)OPTIONALOnly on NO-GO / CAUTION-with-regressions, to file issues. Load /acli then.

Test-user creds, OpenAPI/API_TOKEN, DBHub and Playwright browsers live inside the CI runner, not the orchestrator — this skill does not exercise them locally, so they are out of scope for this gate. After the gate clears (all REQUIRED GREEN), continue to Phase 0 below.


Phase 0 — Session resume check (MANDATORY, inline)

Before suite selection or any gh workflow run, run the resume contract from agentic-qa-core/references/session-management.md §4:

  1. Compute prospective <scope> = <env>-<YYYY-MM-DD> from invocation context (env defaults to {{DEFAULT_ENV}}).
  2. Check .session/regression-testing/<scope>/progress.md.
  3. If it does NOT exist → proceed to suite selection + Phase 1 preflight + plan.md write.
  4. If it DOES exist:
    • Read plan.md (captured suite, env, workflow_file, RUN_ID if Phase 1 already triggered).
    • Read tail of progress.md.
    • If RUN_ID is present AND progress.md last entry is Phase 1 — Trigger — status: completed but Monitor entry is missing/failed: surface the option to re-attach to the existing RUN_ID via gh run view <RUN_ID> --json status,conclusion instead of re-triggering. This is the high-value resume case.
    • Otherwise surface the standard offer resume / restart / abort. On restart, archive to .session/.archive/<YYYY-MM-DD>-regression-testing-<scope>-aborted/ first.

When to run each suite

SuiteWorkflow fileDurationUse when
regressionregression.yml20-60 minPre-release validation, nightly full run
smokesmoke.yml2-5 minPost-deploy health check, @critical only
sanitysanity.yml1-10 minValidate one feature / one file / one grep pattern

If the user says "run regression" with no qualifier, default to regression on {{DEFAULT_ENV}}. If they sa


Content truncated.

When not to use it

  • Writing new regression tests
  • Performing manual fix verification

Limitations

  • Cannot be used for writing new regression tests
  • Cannot be used for manual fix verification
  • Does not allow skipping analysis before generating a report

How it compares

This skill automates the entire release-readiness pipeline from execution to reporting, unlike manual processes that require separate steps for triggering, monitoring, analysis, and reporting.

Compared to similar skills

regression-testing side by side with the closest alternatives in the catalog.

SkillInstallsUpdatedSafetyDifficulty
regression-testing (this skill)018dReviewAdvanced
cicd-diagnostics13moReviewAdvanced
posthog-ci-integration110dCautionIntermediate
verification-quality-assurance12moReviewIntermediate

Try saying

Example prompts that trigger this skill in your AI assistant.

Search skills

Search the agent skills registry