ci

CI

Artifact Download

# Main camera-box binary (Linux amd64)
gh run download --repo zbynekdrlik/camera-box -n camera-box-linux-amd64 --dir ./dist

# Probe tools — Linux (recording-verdict, frame-probe, camera-box-probe)
gh run download --repo zbynekdrlik/camera-box -n probe-tools-linux-amd64 --dir ./probe-bins

# Probe tools — Windows (recording-verdict.exe, obs-watchdog-gate.exe, obs-self-heal-gate.exe —
# the latter two are the #391/#411 OBS liveness-watchdog + self-heal gate binaries, needed on-box
# by scripts/obs-self-heal-install.sh's Task Scheduler recovery script)
gh run download --repo zbynekdrlik/camera-box -n probe-tools-windows-amd64 --dir ./probe-win

gh run download strips the +x bit — always re-add:

chmod +x ./probe-bins/*   # or ./dist/camera-box

Python harness tests (tests/python/)

CI job python-tests runs python -m pytest tests/python -v after pip install pytest websocket-client matplotlib (clean, minimal). These pin the obs_phase2.py pure helpers + arg parsing (no live OBS) — e.g. the _blackcheck_verdict, _diverging_locked_keys, and #328 _rpc_timed_out deadline helper.

Local gotcha: dev1's global pytest has a broken pytest-html plugin (missing jinja2) that aborts collection with ModuleNotFoundError: No module named 'jinja2'. CI is unaffected (its venv has only the three deps above). Locally, disable the plugin:

python3 -m pytest tests/python -q -p no:html -p no:cacheprovider

Discord CI Notifications (#25)

camera-box CI posts failures to Discord via the NewLevelMedia Discord bot REST API, channel #notifications🔔 (ID 1257652233714270219).

Mechanism: POST https://discord.com/api/v10/channels/<CHANNEL_ID>/messages with Authorization: Bot <token> and {content: "..."}. Discord returns HTTP 200 + the created message object — the id field is real delivery proof (unlike a relay "accepted" ack).

GitHub secrets: DISCORD_BOT_TOKEN (sourced from ~/.claude/channels/discord/.env)

• DISCORD_CHANNEL_ID = 1257652233714270219. The notify-on-failure job (needs all gates, if: failure()) skips gracefully (exit 0) if either secret is unset.

DEAD PATH — do NOT reuse: CLAUDE_DISCORD_WEBHOOK_URL (the n8n relay at ~/.claude/env) ACCEPTS POSTs (HTTP 200 Workflow was started) but no longer DELIVERS to Discord (retired). HTTP 200 from it is a liveness ack, NOT delivery.

DM vs notifications channel:

• 1491798759103795301 (DISCORD_NOTIFICATION_CHANNEL_ID in the channels .env) = bot↔user DM channel used by notify-discord.sh idle pings — this is a DM, NOT the shared #notifications channel. Cannot hold webhooks (404 Unknown Channel on the webhooks endpoint).

Lesson: a relay/webhook returning 2xx only means "accepted", never "delivered". Confirm via the created-message id in the response body.

The Discord bot can't fetch_messages on these channels from an in-session Discord plugin (not allowlisted) — verify delivery by the bot REST POST returning a message id.

#711/#719 Discord report routing: scripts/lib/e2e-discord-report.sh prefers DISCORD_NOTIFICATION_CHANNEL_ZBYNEK (the owner's own thread, push-notifies via a <@DISCORD_MENTION_ZBYNEK> prefix on chunk 1) over the plain #notifications channel (DISCORD_CHANNEL_ID, now a logged fallback only — the user does not watch it and Discord never pushes a phone notification without a mention/DM). CI (full-path-e2e.yml) passes only DISCORD_BOT_TOKEN/DISCORD_CHANNEL_ID as GitHub secrets — the owner-thread vars are backfilled from ~/.claude/channels/discord/.env on every send (this runs on the dev1 self-hosted runner, so that file is always present), preserving an already-set DISCORD_BOT_TOKEN so CI's real secret is never shadowed by the local file's own value.

TESTING PATTERN — a fake curl on PATH that records multi-line argv safely. A test that drives a bash function calling curl -d "$payload" ... and wants to assert on the ACTUAL payload sent must capture the fake curl's argv into a log file — but the payload itself (a composed Discord report) is MULTI-LINE, so a newline-delimited log is ambiguous (one arg's embedded newlines look identical to arg boundaries). Use ASCII Record Separator (\x1e) between args and Group Separator (\x1d) between calls instead — bytes that never occur in real curl argv (headers, JSON, URLs):

# fake curl:
{ for a in "$@"; do printf '%s\x1e' "$a"; done; printf '\x1d'; } >> "$CURL_LOG"
printf '{"id":"999"}\n200'   # canned success response, mirrors `curl -w '\n%{http_code}'`

Parse in the test harness by splitting on \x1d then \x1e. See tests/harness_e2e_discord_report_owner_thread_719.rs for the full pattern (PATH-stub convention, same family as tests/harness_deploy_fleet.rs's sshpass/gh stubs).

Probe Binary Flow — Run on stream.lan, NOT dev1

OBS records the 0.7–6 GB program file on stream box (10.77.9.204 — strong CPU, fast disks). Decode WHERE THE VIDEO IS — run recording-verdict on stream against the LOCAL recording; bring back ONLY the small verdict JSON (+ pixel-proof PNGs). dev1 holds NOTHING big.

# 1. Download Windows verdict for the commit under test
gh run download --repo zbynekdrlik/camera-box -n probe-tools-windows-amd64 --dir ./probe-win

# 2. Upload to stream box via win-stream-snv MCP FileUpload:
#    path='C:\camera-box\recording-verdict.exe'

# 3. Run on stream box via win-stream-snv Shell (see scripts/recording-verdict-on-stream.sh)
scripts/recording-verdict-on-stream.sh \
  --verdict-exe 'C:\camera-box\recording-verdict.exe' \
  --out-dir 'C:\camera-box\verdict-out' \
  --stream-rec 'C:\path\on\box\stream-REC.mp4' \
  -- --strih 'C:\path\on\box\strih-REC.mkv' \
     --stream 'C:\path\on\box\stream-REC.mp4' \
     --min-secs 300 --json 'C:\camera-box\verdict-out\verdict.json' \
     --out-dir 'C:\camera-box\verdict-out\pixel-proof'

# 4. Pull back ONLY verdict JSON + pixel-proof PNGs via win-stream-snv FileDownload.

scripts/recording-e2e.sh defaults to this on-stream flow (VERDICT_ON_STREAM=1). VERDICT_ON_STREAM=0 selects the legacy decode-on-dev1 path (kept for boxes without uploaded verdict.exe).

scp/ssh to Windows (stream.lan) was historically believed DENIED on this rig — corrected below (the #703 section further down): plain OpenSSH+password scp/ssh actually WORKS against stream.lan with the targets.md creds, and is now what E2E_EXECUTE_VERDICT=1 uses for real. This section's FileDownload/manual-plan flow (win-stream-snv MCP) is still the fallback for a manual/ workflow_dispatch operator run, or for pulling files above the few-MB point where FileDownload itself breaks (see #701) — for anything sizeable, prefer scp directly. ffmpeg/ffprobe are already installed on stream.lan (winget; ffmpeg 8.0.1 on PATH).

Probe is CI-only locally — never compile --features probe on dev1 (#185)

The shared dev1 target/ has no GC (rust-lang/cargo#5026). Compiling the heavy probe feature locally (qrcode/rqrr/image/drm/lz4_flex + 5 required-features=["probe"] bins) across the day's workers ballooned it to 18GB and filled the disk. The fix: local Tier-0 cheap-checks run DEFAULT features only — cargo clippy --all-targets -- -D warnings (NO --all-features), cargo test --no-run. Probe is compile-checked + built ON CI (ci.yml runs --all-features; #101 C++ gate + #192 probe-tools artifact). See CLAUDE.md "Local Build Policy".

Convention when adding an integration test that imports camera_box::probe::…: gate the whole file with #![cfg(feature = "probe")] (after the //! doc block, before the first use) — exactly like tests/recording_decode.rs. An UNGATED probe test forces the local default-feature clippy/test --no-run to compile the probe feature and re-balloons target/. CI (--all-features) still runs gated tests. tests/local_build_policy_bounds_target.rs enforces both invariants (it FAILS if any probe-using test is ungated or if the CLAUDE.md local gate command block uses --all-features/--features probe).

Gotcha — a clippy lint inside a probe-gated file (src/probe/*) is INVISIBLE to local Tier-0 clippy and only FAILS on CI's Lint job (which runs cargo clippy --all-targets --all-features). Default-feature local clippy never compiles src/probe/*, so a lint there passes locally and red-X's CI — a wasted CI cycle (#312/#324: a clippy::doc_lazy_continuation in recording_segments.rs module-doc passed Tier-0 but failed CI). When you edit ANY src/probe/* file — especially doc comments — eyeball it for clippy patterns before pushing; you cannot lint-verify it locally (running --all-features to check is banned — it re-balloons target/). Recurring doc trap: a [label]: token mid-paragraph (e.g. [`crate::x`]: text) makes pulldown-cmark parse a link-reference definition → the next lines become a doc_lazy_continuation lint; reword to drop the ]: (e.g. (in [`crate::x`] — text) or add a blank line.

Same class, a COMPILE error not a lint — a serde::Serialize (or any trait-bound) that only propagates under --features probe. bin/recording-verdict.rs's NodeVerdict is #[derive(serde::Serialize)], so EVERY field type must be Serialize. When you change a field to hold a lib-crate struct (e.g. #580 changed imag_optical_beat from Option<bool> to Option<OpticalBeatVerdict> so the printers/JSON could report the beat detail), that struct MUST ALSO derive Serialize — else the whole --features probe build breaks with E0277: the trait bound X: serde::Serialize is not satisfied, red-X'ing Lint + Test + Build + Coverage + Windows probe build at once. INVISIBLE to Tier-0: cargo check/clippy on default features never compiles bin/recording-verdict.rs (required-features = ["probe"]), so it passes locally and only fails on CI (one wasted cycle, #580). Before pushing a NodeVerdict (or any `#[deri

Content truncated.