maintainer-ci-ctest

Maintainer CI and CTest Workflow

Use this skill only for iccDEV maintainer-owned infrastructure changes: .github/**, Dockerfile*, CTest registration, CPack and release packaging, sanitizer helper policy, CodeQL/workflow governance, vcpkg release verification, and security automation.

General contributor requests should be redirected to issue or PR descriptions unless an iccDEV maintainer explicitly approved the infrastructure change.

Scope Decision

Choose the smallest maintainer-owned surface that proves the behavior:

Keep contributor code changes separate from maintainer infrastructure commits when practical.

CTest Rules

• check must exist on every platform.
• check and workflow CTest execution must use --no-tests=error.
• Do not add hard-coded CTest suite totals to workflows, docs, or maintainer instructions; keep suite lists descriptive and let CTest discovery report the current total.
• Adding checks inside iccdev-tool-coverage-baseline.sh does not change that count; validate the direct script and ctest -R '^iccdev\.tool-coverage$'.
• If a change touches legacy packed intent decoding or named-color overprint variants, include both .github/scripts/iccdev-applynamedcmm-cli-args-regression.sh and .github/scripts/iccdev-applysearch-cli-args-regression.sh, plus .github/scripts/iccdev-namedcolor-overprint-regression-tests.sh.
• Windows full builds include focused executable regressions, batch-backed suites, dump/profile smoke coverage, shared-export coverage, and PAWG report coverage.
• The comprehensive build matrix must retain MSVC, ClangCL, and MinGW UCRT64 coverage, plus a separate MSVC full CTest gate with warnings treated as errors. Unix must likewise retain the full strict CTest gate.
• Use rg "Total Tests:|currently register|ci[-]tool[-]tests[.]yml" docs .github before PR handoff to catch stale count and workflow-name references.
• Generated-profile count changes must update every explicit assertion source, including Build/Cmake/Testing/CMakeLists.txt, generated-profile workflows, and packaged WASM regression scripts.
• Do not duplicate generated-profile totals in this skill; use the assertion sources as the current truth.
• Windows batch CTest runs must use the disposable Testing copy under the build tree and must not dirty the source Testing/ directory.
• Windows executable tests must receive runtime DLL directories through Build/Cmake/Testing/WindowsRuntimePaths.cmake; do not rely on a developer or runner shell PATH for vcpkg or MinGW runtime DLLs.
• MinGW builds still need UCRT64 bin on the invoking shell PATH because GCC subprocesses such as cc1plus.exe depend on MSYS2 runtime DLLs during build.

Workflow Rules

• Follow .github/instructions/workflow-governance.instructions.md.
• Treat ci-pr-action full as the deterministic core maintainer gate. It runs Unix GCC/Clang Release and Debug builds, exact GCC 15.2 strict Release LTO in the regression container, non-sanitized GCC core tool tests, and Windows. Its tool-test caller excludes pr-extended and ci-infrastructure CTests.
• ci_scope=auto is the default. It selects the full matrix for source, build, and test changes; documentation-only changes use the constrained fast-lane settings. Workflow-only changes keep the core orchestrator limited to setup and finalization while the standalone preflight and risk-analysis workflows provide their required contexts. Container-only changes use those standalone security gates and local container validation.
• Use ci_scope=fast-lane for the exact GCC 15.2 Release LTO and non-sanitized core tool lanes. Fast lane defaults to the latest CTest with Windows disabled; it does not run a Docker verification job.
• Container changes require the local canonical-image build and smoke in docs/regression-container.md; the Docker PR verification job is disabled.
• Apple mobile core changes must keep the dependency-free apple-*-core presets available, align matching apple-*-extended-core presets with SDK and dependency discovery, and keep simulator app capability/gap reporting in Build/AppleMobile synchronized with docs/build.md.
• Do not use || true around profile generation, CTest discovery, regression execution, sanitizer checks, or packaging verification.
• Use least-privilege permissions and credential cleanup.
• Sanitize all GITHUB_STEP_SUMMARY and GITHUB_OUTPUT writes.
• Check out the base ref's .github/scripts sparsely and source its sanitizer helpers for every workflow that executes PR-controlled source.
• Include github.event_name in concurrency keys for workflows that accept both PR and manual-dispatch events, so a manual lane cannot cancel its PR counterpart.
• Trigger shared-concurrency workflows sequentially to avoid canceling your own run. Use ci-pr-action for normal core validation and the standalone path-scoped/manual ci-regression-checks workflow for ASAN/UBSAN coverage.

Local Validation

For focused workflow iteration, start with:

.github/scripts/preflight-safety-checks.sh --fast-lane

This scans changed workflow/script surfaces without running CTest or local CodeQL databases. Run only the nearest feature test during iteration, then use --fast-lane=matlab for MATLAB-only work. Use the full preflight or hosted gates before final handoff.

file <changed-files>
git diff --check
cmake -S Build/Cmake -B build -DENABLE_TOOLS=ON -DENABLE_TESTS=ON -DENABLE_WXWIDGETS=OFF
cmake --build build --parallel "$(nproc)"
ctest --test-dir build -N --no-tests=error
ctest --test-dir build --output-on-failure --no-tests=error

For tool coverage script changes:

ICCDEV_TOOLS_DIR=$PWD/build/Tools \
ICCDEV_TESTING_DIR=$PWD/Testing \
ICCDEV_TEST_OUTDIR=/tmp/iccdev-tool-output \
  .github/scripts/iccdev-tool-coverage-baseline.sh --asan --quick
ctest --test-dir build -R '^iccdev\.tool-coverage$' --output-on-failure

For ci-pr-lint.yml updates, install clang-tidy, clang-tools, and cppcheck, then follow the canonical component-partitioned local reproduction in docs/build.md#maintainer-static-analysis. Verify that every selected component has its own cppcheck and clang-tidy report, and that the combined reports preserve the sum of the component output.

For workflow YAML:

python3 -c "import yaml; [yaml.safe_load(open(p)) for p in ['.github/workflows/<workflow>.yml']]; print('YAML parse OK')"
actionlint -no-color .github/workflows/<workflow>.yml

For Apple mobile core and simulator workflow changes on macOS:

cmake --list-presets=configure -S Build/Cmake | grep -E 'apple-.*(extended-core|core)'
bash .github/scripts/iccdev-apple-simulator-smoke.sh ios
bash .github/scripts/iccdev-apple-simulator-smoke.sh watchos
bash .github/scripts/iccdev-xcode-ctest-smoke.sh

Set ICCDEV_APPLE_CORE_FLAVOR=minimal only when explicitly validating the dependency-free app path. The default simulator smoke should use the extended core and require the app's JSON report to include built-library checks, the public invalid-profile substitution control, and non-failing mobile gap notes.

For CPack, install/export, vcpkg, or release packaging changes, run the nearest packaging smoke test and inspect logs for missing files, duplicate install manifest entries, CRT mismatch warnings, and skipped smoke coverage.

For Dockerfile* changes:

docker build -t iccdev-container-check -f <Dockerfile> .
docker run --rm iccdev-container-check <smoke-command>

For the unified Dockerfile, follow the complete local development-environment preflight in docs/regression-container.md#maintainer-preflight-and-security-checks. Cached builds are permitted for development iterations. The final pre-push proof requires workflow and Dockerfile policy checks, a no-cache build, analyzer inventory and runtime smoke, a healthy image, and Trivy vulnerability/secret triage. If the image is published, pass the published branch or SHA tag to ci-iccdev-tool-tests.yml.

GitHub Validation

After pushing, trigger only the workflows affected by the change:

gh workflow run "ci-pr-action" --repo InternationalColorConsortium/iccDEV --ref <branch> -f ci_scope=full
gh workflow run "ci-pr-action" --repo InternationalColorConsortium/iccDEV --ref <branch> \
  -f ci_scope=fast-lane -f pr_number=<open-pr-number>
gh workflow run "ci-risk-analysis" --repo InternationalColorConsortium/iccDEV --ref <branch> \
  -f analysis_target="Specific git ref" -f git_ref=<full-sha> -f severity_threshold=HIGH -f fail_on_findings=true

Wait for shared-concurrency workflows one at a time. Capture run IDs, head SHA, job conclusions, artifact names, and key sentinel lines such as Total Tests, `1

Content truncated.