visual-evidence

Visual Evidence Standard

Every change that alters what a user sees or how the UI behaves ships with two things, in the same PR:

1. A regression test — a Playwright spec under test/visual/*.spec.js that would fail if the change regressed.
2. Before/after visual evidence — labelled montages + an overflow/metrics readout generated by test/visual/evidence-kit.mjs, committed under test/visual/evidence/<slug>/ and linked from CHANGELOG.md.

This is what makes a fix auto-merge-eligible (see the autonomy policy in continuous-evolution.md) and what carries the proof into the release notes. The navbar fix (PR #215, test/visual/navbar-evidence.mjs) is the worked reference.

When to use

• Any change touching _sass/**, _includes/**, _layouts/**, assets/**, or any user-visible behaviour (a JS interaction, a responsive breakpoint).
• Right after a bug fix that changed rendering — capture the bug (before) and the fix (after) in the same run.
• Before labelling a fix PR auto-merge.

Opt-out: a change under those paths with no visual/behavioural effect (a comment, a refactor with identical output, an aria-label tweak) is exempt — add the skip-evidence label so the evidence gate passes, and say why in the PR.

The flow

1. Write (or extend) the regression test

A test/visual/*.spec.js that pins the new behaviour and fails on regression. Prefer behavioural/structural assertions (geometry, classes, computed style, overflow) over pixel snapshots so it survives content edits. Reuse the helpers in test/visual/fixtures.js (measureNavbarLayout, NAV_WIDTHS, boxesOverlap, …). Add it to the smoke tier so CI runs it.

2. Generate before/after evidence

Write a short spec for the kit (or pass a config) describing the scenario:

// test/visual/<slug>-evidence.mjs
import { generateEvidence } from './evidence-kit.mjs';
await generateEvidence({
  slug: '<slug>',
  base: process.env.BASE_URL || 'http://localhost:4000',
  route: '/',                 // page that exercises the change
  // unfixCss reverts the fix so the kit reproduces the BEFORE state on one
  // server — omit it for a new feature (after-only evidence).
  unfixCss: `/* CSS that undoes this fix */`,
  widths: [320, 390, 768, 992, 1280, 1440],
  configs: [{ key, label, apply: () => {/* DOM variation */} }],
  title: '<Human title>',
});

Run it against the live dev server:

docker compose up                                    # serves :4000
BASE_URL=http://localhost:4000 node test/visual/<slug>-evidence.mjs

This writes test/visual/evidence/<slug>/: numbered montages, metrics.json, and CHANGELOG-snippet.txt. Open the PNGs and look before committing.

Add a one-paragraph test/visual/evidence/<slug>/README.md explaining the bug and what each image shows (see PR #215's evidence README for the shape).

3. Link evidence into the changelog

Paste the generated CHANGELOG-snippet.txt line into your CHANGELOG.md [Unreleased] entry, so release-please carries the evidence link into the GitHub Release notes. Example:

- **Navbar no longer cut off at narrow widths** … (evidence:
  [`test/visual/evidence/navbar/`](test/visual/evidence/navbar/README.md) —
  page overflow 653px → 0 across 8 widths).

4. Record anything the fix uncovered

If, while fixing, you find a separate issue (the way the navbar fix surfaced the version.rb↔Gemfile.lock drift), do not silently fix it here — file it so the loop owns it. Append a task to _data/backlog.yml (source: issue, summary referencing this PR), bump meta.next_id:

  - id: T-0NN
    title: "<what's broken>"
    status: open
    priority: P1
    area: <tests|infra|a11y|perf|…>
    risk: low            # low + tests+evidence ⇒ the loop can auto-fix & auto-merge
    effort: S
    source: issue
    summary: "Discovered during #<this-PR>: <detail>."
    acceptance:
      - "<checkable criterion>"
      - "regression test added"
      - "before/after evidence committed"
    links: { issue: null, pr: null, roadmap: null }
    created: <YYYY-MM-DD>
    updated: <YYYY-MM-DD>

On merge, sync.yml opens the GitHub Issue and the IMPLEMENT routine picks it up — closing the loop automatically.

5. Validate, commit, open the PR

Run the validate-build skill, then follow change-workflow. Stage evidence by path. The PR should contain: the change, the spec, test/visual/evidence/<slug>/, and the CHANGELOG entry.

Auto-merge a fix (when eligible)

Add the auto-merge label only when ALL hold (the evidence gate + CI enforce the rest; auto-merge.yml re-checks risky files):

• risk: low — no change to public API, version.rb, gemspec, a dependency manifest, or a data schema; no new runtime dependency.
• A passing regression test and committed before/after evidence are in the PR (or skip-evidence is justified for a non-visual fix).
• All acceptance criteria / CI checks green.

gh pr edit --add-label auto-merge

feat, refactor, or anything risk: standard stays for human review.

Pre-PR checklist

• Regression test added/extended under test/visual/*.spec.js (smoke tier).
• test/visual/evidence/<slug>/ committed (montages + metrics + README).
• Evidence link pasted into the CHANGELOG.md [Unreleased] entry.
• Any issue the fix uncovered filed in _data/backlog.yml.
• validate-build green; PR title is a clean Conventional Commit.
• auto-merge label only if the eligibility rules above hold.