validate-screenshots

Validate screenshots

Runs a design-system review on PNG screenshots a content developer added to an OutSystems docs article, so they can self-check before asking for manual design review.

The rubric lives in visual-rules-screenshots.md next to this file — it is produced by running scripts/calibrate.py against a reference set of approved screenshots. If that file is still a placeholder, stop and tell the user to run calibration first (see the Calibration section below).

Setup

Read visual-rules-screenshots.md from the same directory as this SKILL.md. If it still contains the placeholder marker (<!-- NOT CALIBRATED -->), stop and print:

"No screenshot rubric yet. Run scripts/calibrate.py once with a set of approved screenshots to generate visual-rules-screenshots.md, then try again."

Step 1: Resolve targets

Call the target collector with whatever the user passed (default to empty):

python3 -B <skill-dir>/scripts/collect_targets.py "$ARGUMENTS"

It prints a JSON array to stdout. Each entry has:

• image_path — absolute path to the PNG
• article_path — the markdown that references it, or null
• shared_with — only present on a single-image invocation when the calling workflow already knows the image is referenced by more than one article changed in this same pull request; a list of those articles' absolute paths. Mutually exclusive with article_path being set — see "Choosing the summary title" in Step 3.

If the array is empty, stop with:

"No screenshots to validate. Pass a markdown file, a PNG path, or run without arguments on a branch that has image changes vs. master."

Duplicate detection

After collecting targets, group entries by their sha256 field. If two or more entries share a hash, the files are byte-for-byte identical and only one copy should exist. Mark every duplicate with a ❌ finding in the final summary:

"Byte-identical duplicate of <other image name>. Keep one and delete the rest."

Empty sha256 values (file unreadable) are not grouped.

Run this over every entry, including ones marked "cached": true. Whether an image duplicates another depends on the whole set, not on that image's own content, so a cached image can become a duplicate when a new copy is added and stop being one when the copy is deleted. This grouping is cheap and needs no vision, so it is never skipped. Note sha256 is used here and nowhere else; the cache keys on blob_sha instead.

Step 2: Validate each target

Cached entries first. If an entry has "cached": true, this exact file content already passed this rubric on an earlier run. Skip the numbered steps below for it entirely — no Read into vision, no checker scripts — and reuse its cached_findings array verbatim as that image's findings (an empty array means it passed clean). It still counts toward N in Step 3, and toward M when cached_findings is non-empty. Do not re-word, re-order, or re-judge a cached finding: it was produced by this same rubric and must read identically so the report stays stable across runs.

cache_reason and cache_detail on each entry say why it was or wasn't cached; they are for the log, not for the report.

For every entry that is not cached:

1. Read the image with the Read tool so it goes into vision.

2. Run the deterministic checkers. They are authoritative for the rules they cover — do not second-guess them from vision.

   python3 -B <skill-dir>/scripts/check_metadata.py "<image_path>"
   python3 -B <skill-dir>/scripts/has_shadow.py    "<image_path>"
   python3 -B <skill-dir>/scripts/check_red.py     "<image_path>"
   

   • check_metadata.py prints JSON covering rule 1 (PNG format), rule 2 (filename regex + surface suffix, including double-suffix detection), and rule 10 (width ≤ 1200 px). Parse the JSON and use each verdict directly.
   • has_shadow.py prints one of shadow: true / shadow: wrong / shadow: false / shadow: inconclusive / shadow: unknown for rule 6. Map them to severity as follows: true passes. wrong is ❌ — output exactly this text (do not paraphrase): "Shadow doesn't match the TK-shadow effect available in the TK design library — apply it from the Effects menu in Figma (rule 6)". false is ❌ for large-surface captures and ⚠️ otherwise — never a clean pass, so the reviewer always sees it and can dismiss when it's genuinely a self-bounded close crop. Output exactly this text (do not paraphrase): "Missing TK-shadow effect available in the TK design library — apply it from the Effects menu in Figma (rule 6)". inconclusive is ⚠️ — the alpha channel is present but the detector couldn't reach opaque content on the sampled edges (e.g. content far from the borders, transparent mid-edges). Output exactly this text (do not paraphrase): "Shadow check inconclusive — designer should verify the TK-shadow effect available in the TK design library (Effects menu in Figma) is applied (rule 6)". Do not restate it as "missing shadow" — the script didn't say that. unknown means the image has no alpha channel at all (typically a JPG saved as .png, which rule 1 already fails); don't emit a separate shadow finding in that case. Do not suppress a false verdict based on your own close-crop judgment — surface it as ⚠️ at minimum. Refer to rule 6 in the rubric for the full definitions.
   • check_red.py prints one of red: ok #RRGGBB / red: wrong #RRGGBB (token #F22800) / red: none for the color half of rule 3. Map it as: ok passes; none passes (the image has no red highlight, so there's nothing to color-check); wrong is ⚠️ — phrase as "Highlight red is #RRGGBB, not the design-system token #F22800 — re-snap from the Figma library", quoting the actual hex from the script so the user can confirm. Vision can't reliably distinguish similar reds (#F22800 vs #CC2200 vs #BB1F00), so do not second-guess the script.

3. Use vision only for the rules the scripts don't cover: rule 2's suffix-vs-content mismatch (filename says -ss but the image shows ODC Studio), rule 3's placement (whether a highlight is present and on the right element — the color is scripted; check visual-rules-screenshots.md rule 3's "Native selection state" case first — if the product's own selection highlight already marks the focal element but no red rectangle was added, that's a ⚠️ with the quoted verdict wording, not a ❌; only fail ❌ when there's neither a red rectangle nor a native selection state), rule 4 (numbered callouts), rule 5 (arrows), rule 7 (PII), rule 8 (internal environment URLs).

   For -ss/-odcs files showing an entity or data-model layout (boxes connected by lines), check against visual-rules-screenshots.md rule 2's entity/data-model carve-out before concluding the suffix is wrong. A Data-tab capture cropped tight with no toolbar chrome is not by itself a sign of a misfiled -diag diagram — don't fall back on the genuine-diagram checklist below as the only reference point for that judgment.

   For files ending in -diag: use vision to determine whether the content is a genuine diagram or a UI screenshot.

   A genuine diagram has all of the following:

   • No product UI chrome (no IDE toolbars, portal navigation, or browser address bar)
   • A conceptual layout: flow chart, sequence diagram, hierarchy tree, architecture box, icon grid, or swimlane diagram
   • OutSystems branded orange/red iconography representing concepts
   • Connecting elements: arrows, dashed lines, or swimlanes
   • A white or light-gray background with a rounded-rectangle shadow border
   • Descriptive text labels naming concepts or roles, not UI controls

   Excluded: entity or data-model diagrams captured directly from Service Studio's or ODC Studio's Data tab, even when cropped tight with no visible toolbar chrome. These are native tool renderings, not standalone Figma diagrams — refer to visual-rules-screenshots.md rule 2's entity/data-model carve-out for the native iconography and connector traits that identify them.

   A UI screenshot shows product interface chrome with interactive elements (panels, toolbars, dropdowns, form fields, menus) from ODC Studio, ODC Portal, Service Studio, or similar tools.

   • If the content is a genuine diagram → suffix is correct; no naming finding.
   • If the content is a UI screenshot → emit ❌ Rule 2: "Image looks like a UI screenshot but uses the -diag suffix — rename with the correct surface suffix (e.g. -ss, -odcs)."

4. Collect only failures and warnings — passing rules are not reported.

Keep per-image notes internal until every image is processed. Do not stream partial reports.

Extract the Figma link (once per article)

If any entry has a non-null article_path, open that markdown file once and grep its YAML frontmatter for a figma: key (the value is a figma.com URL). Remember the value — the report header will surface it so the content developer can jump straight to Figma to fix things. If the frontmatter has no figma: field, or the value is empty, skip the link entirely (don't print an empty Figma: line).

When multiple articles reference different screenshots (branch-diff mode), collect the unique Figma URLs per article and emit them grouped under each article in the header. In the common single-article case, one line is enough.

Step 3: Emit the summary

Verdict block (always, before anything else)

Print one line recording the outcome for every target in this invocation — cached and freshly validated alike — before any other Step 3 output, and before the <!-- REPORT BEGIN --> sentinel the calling workflow expects:

<!-- VERDICTS {"src/foo/images/a-ss.png":

---

*Content truncated.*