regression-testing
Orchestrates regression test pipelines and provides actionable release-readiness insights.
Install
mkdir -p .claude/skills/regression-testing && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/18122" && unzip -o skill.zip -d .claude/skills/regression-testing && rm skill.zipInstalls to .claude/skills/regression-testing
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Execute regression test suites via CI/CD, analyze results, classify failures, and produce GO/NO-GO release decisions. Use when running regression, smoke, or sanity suites through GitHub Actions, monitoring workflow runs, downloading Allure or Playwright artifacts, classifying failures (REGRESSION vs FLAKY vs KNOWN vs ENVIRONMENT vs NEW TEST), computing pass-rate and trend metrics, deciding release readiness, generating executive quality reports, or creating regression issues. Triggers on: run regression, trigger test workflow, analyze test results, quality report, GO/NO-GO decision, release readiness, flaky tests, Allure report, smoke suite, pass rate, nightly test failure, stage 6. Do NOT use for writing new regression tests (that belongs to test-automation) or for manual fix verification (that belongs to sprint-testing).Key capabilities
- →Trigger CI suite runs through GitHub Actions
- →Monitor CI workflow runs to completion
- →Classify test failures into categories like REGRESSION or FLAKY
- →Compute pass-rate and trend metrics for test results
- →Generate executive quality reports for stakeholders
- →Create regression issues in Jira based on analysis
How it works
The skill orchestrates the release-readiness pipeline by triggering a CI suite, monitoring its completion, classifying failures, and scoring against release criteria. It then emits a verdict and a stakeholder report.
Inputs & outputs
When to use regression-testing
- →Trigger smoke test suites
- →Analyze nightly test failures
- →Generate quality reports for stakeholders
- →Classify flaky vs. regression tests
About this skill
Forbidden invocations
NEVER invoke /sdd-* skills from this workflow. SDD is an optional
user-installed ceremony; this skill ships self-contained and does not chain
SDD under any condition. If you need to refactor KATA, fixtures, cli/,
scripts/, or api/schemas/ pipeline, exit this skill first and invoke
/framework-development — which itself runs Plan → Code → Verify → Archive
natively (no SDD required).
This boundary is mechanical, not advisory: scripts/lint-skills.ts rejects
any /sdd- mention outside this section. See:
.claude/skills/agentic-qa-core/references/skill-composition-strategy.md §4
(governs users who manually install SDD).
Regression Testing — Execute, Analyze, Decide
Orchestrates the full release-readiness pipeline: trigger a CI suite, monitor it to completion, classify failures, score against release criteria, and emit a GO / CAUTION / NO-GO verdict plus a stakeholder report.
Three phases, always in this order: Execute → Analyze → Report. Do not skip analysis and jump to a report. Do not guess classification without reading failure logs.
Inputs
.github/workflows/*.yml— workflow files for regression / smoke / sanity suites; defines triggers, inputs, and artifact uploads..context/master-test-plan.md— regression Epic key + expected pass-rate SLOs per suite.playwright.config.ts— reporter config, retry policy, project matrix; needed to interpret retry counts and shard splits.- Previous run's Allure report (artifact URL or local download under
./analysis/previous/) — baseline for trend computation. kata-manifest.json— registry of tests and ATCs available; used to cross-reference failed test IDs..agents/jira-required.yaml— Jira refs (project key, work types, transitions) for filing regression issues.agentic-qa-core/references/defect-management-doctrine.md— canonical authority for classifying (Bug/Defect/Improvement), the mandatory field matrix, QA-Assignee ownership, and the QA process epic when a confirmed regression is filed in Jira (Phase 3). Read BEFORE filing any defect.
Subagent Dispatch Strategy
Orchestration & Session contracts: this skill follows
./orchestration-doctrine.md(mandatory subagent dispatch — main thread is command center) AND./session-management.md(Phase 0 resume check, plan-first persistence at.session/<skill-slug>/<scope>/, archive on completion). Phase 0 (resume check) and Phase 1 (plan write) are NOT optional. The orchestrator also applies the per-stage Definition-of-Done gates in./stage-gates.md: verify a stage's DoD BEFORE recording its progress checkpoint and advancing.
This skill is per-run scope: <scope> = <env>-<YYYY-MM-DD> (e.g. staging-2026-05-20). Session state lives at .session/regression-testing/<scope>/{plan.md, progress.md} per agentic-qa-core/references/session-management.md §3 + §9. The single highest-value resume case: if the Monitor subagent dies while watching a long CI run but RUN_ID was captured in plan.md, Phase 0 re-attaches via gh run view <RUN_ID> instead of re-triggering CI (saves 20–60 min of wall-clock).
This skill is compliant with the doctrine in CLAUDE.md §"Orchestration Mode (Subagent Strategy)" and the session contract in .claude/skills/agentic-qa-core/references/session-management.md. Every dispatch follows the 6-component briefing format defined in .claude/skills/agentic-qa-core/references/briefing-template.md, and the pattern selected per stage matches the decision guide in .claude/skills/agentic-qa-core/references/dispatch-patterns.md. The two CI-bound stages (long-running watch, multi-artifact download) and the high-volume failure classification step are the hotspots — everything else stays inline because the dispatch overhead is not justified.
| Stage | Pattern | Subagent role |
|---|---|---|
Trigger workflow (gh workflow run) | Single | inline — no dispatch needed (one shell call) |
Wait/monitor gh run watch | Background | one Monitor subagent runs the watch; main thread continues with prep work; subagent notifies on exit |
| Download 3 artifacts (allure / evidence / playwright) | Parallel | 3 simultaneous subagents, one per artifact; cap = 3 (no rate-limit risk) |
| Classify failures (chunks of ~10 tests each) | Parallel | N subagents based on failure volume; cap = 10 to avoid context dilution |
| Compute metrics (pass-rate, trends) | Single | inline — needs aggregated state, low cost |
| Generate executive report | Single | inline — final synthesis, decisions live here |
| GO / CAUTION / NO-GO verdict | Single | inline — main thread owns release decisions |
- Error protocol: On any subagent failure: STOP, report full context to user, present retry / skip / abort options. Do NOT auto-fix. See
.claude/skills/agentic-qa-core/references/orchestration-doctrine.md.
Readiness Preflight Gate (MANDATORY — runs before Phase 0)
Full doctrine:
agentic-qa-core/references/preflight-gate.md. Runs FIRST, before the resume check and anygh workflow run. Two laws: (1) args-as-answers — the suite (regression/smoke/sanity), env, and any grep/test_file are provided args; ask only the gaps. (2) probe, don't assume. Surface gaps + REDs as ONEAskUserQuestionchecklist; self-fix with approval + explanation; STOP on any blocking RED. This generalizes the Phase 1 §Preflight (gh auth) to a full readiness check pulled to t=0. Generic baseline (env resolution, secret/restart handling, the two laws, output contract) is inherited from the reference §3.1 — not repeated here. Below is only this skill's specific capability delta (note: test-user creds, MCPs and browsers live inside the CI runner, not the orchestrator).
| Capability | Need | Why here |
|---|---|---|
| GitHub CLI authenticated | REQUIRED | Every stage drives CI via gh (gh auth status, gh workflow run, gh run watch, gh run download). Not authed → user runs gh auth login (suggest the ! prefix); do not proceed. |
| Workflow files present | REQUIRED | .github/workflows/ must hold the regression/smoke/sanity workflow for the chosen suite, with the inputs this skill passes. |
| GitHub Actions Secrets/Variables | REQUIRED | The runner authenticates with env-prefixed creds (secrets.<ENV>_USER_EMAIL / _PASSWORD) + XRAY_* / ATLASSIAN_* as Repository/Environment Secrets — the suite 401s mid-run without them. gh secret list (add --env <env> for environment scope) shows them; missing → gh secret set <NAME> from .env. /adapt-framework only emits a manual list today, so this is the most common silent gap. |
| Allure 3 local | REQUIRED | bunx allure resolves (devDep, no global install); allurerc.mjs present for bun allure:agent markdown triage. |
| Active env | REQUIRED | The suite runs against <<ACTIVE_ENV>> (default {{DEFAULT_ENV}}). Confirm it is the intended target before a 20–60 min run. |
[TMS_TOOL] (result sync) | OPTIONAL | Only when .agents/project.yaml testing.tms_cli is set — Stage 3 pushes run status. jira-xray → /xray-cli + XRAY_*. |
[ISSUE_TRACKER_TOOL] (file regression issues) | OPTIONAL | Only on NO-GO / CAUTION-with-regressions, to file issues. Load /acli then. |
Test-user creds, OpenAPI/API_TOKEN, DBHub and Playwright browsers live inside the CI runner, not the orchestrator — this skill does not exercise them locally, so they are out of scope for this gate. After the gate clears (all REQUIRED GREEN), continue to Phase 0 below.
Phase 0 — Session resume check (MANDATORY, inline)
Before suite selection or any gh workflow run, run the resume contract from agentic-qa-core/references/session-management.md §4:
- Compute prospective
<scope>=<env>-<YYYY-MM-DD>from invocation context (env defaults to{{DEFAULT_ENV}}). - Check
.session/regression-testing/<scope>/progress.md. - If it does NOT exist → proceed to suite selection + Phase 1 preflight + plan.md write.
- If it DOES exist:
- Read
plan.md(capturedsuite,env,workflow_file,RUN_IDif Phase 1 already triggered). - Read tail of
progress.md. - If
RUN_IDis present ANDprogress.mdlast entry isPhase 1 — Trigger — status: completedbut Monitor entry is missing/failed: surface the option to re-attach to the existingRUN_IDviagh run view <RUN_ID> --json status,conclusioninstead of re-triggering. This is the high-value resume case. - Otherwise surface the standard offer resume / restart / abort. On
restart, archive to.session/.archive/<YYYY-MM-DD>-regression-testing-<scope>-aborted/first.
- Read
When to run each suite
| Suite | Workflow file | Duration | Use when |
|---|---|---|---|
regression | regression.yml | 20-60 min | Pre-release validation, nightly full run |
smoke | smoke.yml | 2-5 min | Post-deploy health check, @critical only |
sanity | sanity.yml | 1-10 min | Validate one feature / one file / one grep pattern |
If the user says "run regression" with no qualifier, default to regression on {{DEFAULT_ENV}}. If they sa
Content truncated.
When not to use it
- →Writing new regression tests
- →Performing manual fix verification
Limitations
- →Cannot be used for writing new regression tests
- →Cannot be used for manual fix verification
- →Does not allow skipping analysis before generating a report
How it compares
This skill automates the entire release-readiness pipeline from execution to reporting, unlike manual processes that require separate steps for triggering, monitoring, analysis, and reporting.
Compared to similar skills
regression-testing side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| regression-testing (this skill) | 0 | 18d | Review | Advanced |
| cicd-diagnostics | 1 | 3mo | Review | Advanced |
| posthog-ci-integration | 1 | 10d | Caution | Intermediate |
| verification-quality-assurance | 1 | 2mo | Review | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
You might also like
cicd-diagnostics
dotCMS
Diagnoses DotCMS GitHub Actions failures (PR builds, merge queue, nightly, trunk). Analyzes failed tests, root causes, compares runs. Use for "fails in GitHub", "merge queue failure", "PR build failed", "nightly build issue".
posthog-ci-integration
jeremylongshore
Configure PostHog CI/CD integration with GitHub Actions and testing. Use when setting up automated testing, configuring CI pipelines, or integrating PostHog tests into your build process. Trigger with phrases like "posthog CI", "posthog GitHub Actions", "posthog automated tests", "CI posthog".
verification-quality-assurance
ruvnet
Comprehensive truth scoring, code quality verification, and automatic rollback system with 0.95 accuracy threshold for ensuring high-quality agent outputs and codebase reliability.
circleci-why-flaky
mui
Classify recent CircleCI workflow failures into buckets so you can see what's flake vs what's a real bug. Use when CI on a branch is failing repeatedly and you want to know whether retries will fix it. Infers project + branch from the current git repo; defaults to last 7 days.
ci
PioneersHub
Run the local CI pipeline (ruff, bandit, pytest, sonar-scanner) and refresh SonarQube.
qa-tester
svilupp
Browser automation QA testing skill. Systematically tests web applications for functionality, security, and usability issues. Reports findings by severity (CRITICAL/HIGH/MEDIUM/LOW) with immediate alerts for critical failures.