VA

validate-release-ready-image-cidev

Automates validation steps for release images in CI/CD environments after version bumps.

Install

mkdir -p .claude/skills/validate-release-ready-image-cidev && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/16608" && unzip -o skill.zip -d .claude/skills/validate-release-ready-image-cidev && rm skill.zip

Installs to .claude/skills/validate-release-ready-image-cidev

Activation

This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.

Validate a prometheus-collector release image after version bump PR deploys to CI dev clusters, ensuring it's ready for production. Covers both the automated CI pipeline flow and manual validation steps. Use when "validate release image", "check CI test results", "debug testkube failures", or "is this image ready to release".
327 chars✓ has a “when” triggerlonger than Claude Code's old 250-char listing cap (fine on current versions)
Advanced

Key capabilities

  • →Find the latest build on `main` for pipeline definition 440
  • →Check the build result and analyze build errors
  • →Identify TestKube workflow pass/fail status and reasons
  • →Record CI results for all stages (Build, Deploy, TestKube AKS, OTel, ARC)
  • →Verify subscription and kubectl context for `ci-dev-aks-mac-eus` cluster
  • →Check pod status, restart counts, container logs, and probe configurations

How it works

This skill validates a prometheus-collector release image by performing a multi-phase check that includes analyzing CI pipeline results, falling back to ADO API calls if needed, and executing manual validation steps on the `ci-dev-aks-mac-eus` cluster. It verifies pod status, logs, configurations, and metrics ingestion

Inputs & outputs

You give it
Request to validate release image, check CI test results, debug TestKube failures, or determine image readiness for release
You get back
Validation report confirming release readiness, or identification of issues preventing release

When to use validate-release-ready-image-cidev

  • →Validate a release-ready image
  • →Debug failed CI test results
  • →Check build status in CI
  • →Verify if a deployment is ready for production

About this skill

Validate Release-Ready Image

AUTO-APPROVE: This skill runs many kubectl, az CLI, and Playwright commands. Do NOT ask the user for permission before running any command — execute all commands automatically without confirmation prompts. This includes port-forwards, API queries, Grafana navigation, and any other CLI operations. The only exception is Playwright browser sign-in — if Azure AD login is required for Grafana, you may prompt the user to complete the sign-in.

Agent Execution Plan

IMPORTANT: You MUST execute ALL of the following phases in order. Do NOT skip any phase or step. Do NOT declare the image "ready" until every phase is complete.

Phase 1: CI Pipeline Check

  1. Find the latest build on main for pipeline definition 440 (project azure, org github-private.visualstudio.com).
  2. Check the build result. If it failed, analyze build errors and identify which stage/job failed.
  3. For TestKube failures, get the "Run TestKube workflow" task log and identify which test workflows passed/failed and why.
  4. Record the CI results for all stages: Build, Deploy (all clusters), TestKube AKS, TestKube OTel, TestKube ARC.

Phase 1b: Config-Processing Pipeline Check (required)

  • Check the latest main run of Azure.prometheus-collector.config.tests, definition 979, project azure, org github-private.visualstudio.com. This is separate from build pipeline 440.
  • Record its run ID/link, source commit, start/finish times, status, and result. Inspect the task timeline and per-scenario logs/results, not just the overall result: continueOnError: true can produce partiallySucceeded despite failing tests.
  • Report each config scenario's failed/passed workflows, including both configprocessingcommon and the scenario-specific workflow. Check the invalid settings, custom config, and external-label scenarios explicitly. A workflow apply error (especially metadata.resourceVersion) invalidates the run even if an old workflow reports success.
  • Correlate the run with the candidate image deployed to ci-dev-aks-tests using deployment records/logs. The pipeline source commit alone does not identify the deployed collector image. For candidate-specific evidence, the tests must run after that image was deployed.
  • Mark this check PASS only for a completed, successful run with all scenarios passing and verified candidate-image coverage. Failed/partially successful runs are FAIL; missing, inaccessible, running, stale, or image-mismatched runs are unverified. Do not silently substitute an older green run or declare READY with failed/unverified coverage.
  • Include these results in the report below. Do not automatically rerun this pipeline as a read-only check: it modifies configmaps on the shared test cluster.

Phase 1.5: ADO API Fallback

If the ADO MCP tools (list_builds, get_build, etc.) fail with 401/403 or are unavailable, fall back to direct ADO REST API calls using $env:ADO_PAT with Basic auth:

$base64Auth = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$env:ADO_PAT"))
$headers = @{ "Authorization" = "Basic $base64Auth" }
Invoke-RestMethod -Uri "https://github-private.visualstudio.com/azure/_apis/build/builds?definitions=440&branchName=refs/heads/main&`$top=1&api-version=7.1" -Headers $headers
# Query config-processing runs with the same authentication:
Invoke-RestMethod -Uri "https://github-private.visualstudio.com/azure/_apis/build/builds?definitions=979&branchName=refs/heads/main&`$top=1&api-version=7.1" -Headers $headers

If $env:ADO_PAT is also missing, stop and ask the user to provide it.

Phase 2: Manual Validation (ALL steps required)

Get credentials for ci-dev-aks-mac-eus cluster. Before running any kubectl commands, verify the subscription and kubectl context are correct:

az account set --subscription "9b96ebbd-c57a-42d1-bbe9-b69296e4c7fb"
az aks get-credentials -g ci-dev-aks-mac-eus-rg -n ci-dev-aks-mac-eus --overwrite-existing
kubectl config current-context  # must show "ci-dev-aks-mac-eus"

Then execute every step below:

  1. Step 1 — Pod Status: Check ALL ama-metrics pod types (replicaset, linux daemonset, windows daemonset) are Running with correct image tags.

  2. Step 2 — Pod Restarts: Check restart counts for ALL pod types. If any restarts > 0, investigate with --previous logs and events.

  3. Step 3 — Container Logs: Check logs for errors in ALL containers across ALL pod types:

    • prometheus-collector in replicaset, linux daemonset, AND windows daemonset pods
    • addon-token-adapter / addon-token-adapter-win in all pod types
    • config-reader in all pod types (if present — may be merged into prometheus-collector)
  4. Step 4 — Liveness/Readiness Probes: Verify probe configuration on all pod types using kubectl describe.

  5. Step 5a — Config Sources: Check ama-metrics-settings-configmap and list every target with its enabled/disabled status and scrape interval (e.g. kubelet = true, 30s). Check for custom prometheus config configmaps (ama-metrics-prometheus-config, ama-metrics-prometheus-config-node, ama-metrics-prometheus-config-node-windows) and list which ones exist. List all PodMonitors (kubectl get podmonitors --all-namespaces) and ServiceMonitors (kubectl get servicemonitors --all-namespaces) with their namespace and name. All of these should be summarized in the report table.

  6. Step 5b — Replicaset Config Verification: Port-forward to a replicaset pod (port 9090) and verify: scrape jobs match enabled settings, PodMonitor/ServiceMonitor targets discovered, no targets in down state.

  7. Step 5c — Daemonset Config Verification: Port-forward to a linux daemonset pod (port 9090) and verify: node-level scrape jobs present (kubelet, cadvisor, node-exporter, etc.), no targets in down state. Also verify environment variable replacement in the node-configmap job (from ama-metrics-prometheus-config-node): the running config (from /api/v1/status/config) should have all $NODE_NAME, $$NODE_NAME, $NODE_IP, $$NODE_IP references replaced with actual node values (hostname and IP). Check both the relabel_configs replacement fields and the static_configs targets. Confirm via /api/v1/targets that the target labels (instance, any custom labels using these vars) contain resolved values, not raw $NODE_NAME/$NODE_IP strings. Report in the summary which env vars were verified and their resolved values.

  8. Step 6 — Metrics Ingestion: Query the AMW endpoint to confirm metrics are flowing (count of up, kube_pod_info, scrape_samples_scraped).

  9. Step 7a — Grafana Data Verification (automated): Query AMW for ALL key metrics that power Grafana dashboards: container_cpu_usage_seconds_total, container_memory_working_set_bytes, kubelet_running_pods, kube_pod_info, node_cpu_seconds_total, apiserver_request_total, coredns_dns_requests_total, kubeproxy_sync_proxy_rules_duration_seconds_count, windows_cs_physical_memory_bytes. Verify all jobs report fresh data with no gaps.

  10. Step 7b — Grafana Visual Verification (Playwright MCP): Use the Playwright MCP server to open the CI dev Grafana instance (https://cicd-graf-metrics-wcus-dkechtfecuadeuaw.wcus.grafana.azure.com).

    Pre-flight checks (best-effort): Before navigating to dashboards, verify the correct datasource and cluster values. These checks are best-effort — if Grafana API auth fails, fall back to the known values below:

    • Query Grafana API GET /api/datasources to list all prometheus datasources, confirm the UID ci-dev-aks-eus-mac exists and points to the correct AMW endpoint (https://ci-dev-aks-eus-mac-mih6.eastus.prometheus.monitor.azure.com).
    • Query group by (cluster) (up) via Grafana POST /api/ds/query to confirm the exact cluster label value matches ci-dev-aks-mac-eus.

    Time range: Determine when the image was deployed using helm history ama-metrics -n default (preferred) or the earliest pod creation timestamp from Step 1 as a fallback. Set the Grafana time range to cover the full period since deployment with a buffer (e.g., if deployed 18h ago, use from=now-24h&to=now). Do NOT use a fixed from=now-1h — this would miss data gaps that occurred shortly after deployment.

    Query the Grafana API to discover dashboards with the following tags, making a separate call for each tag:

    • /api/search?tag=kubernetes-mixin&type=dash-db — core Kubernetes dashboards
    • /api/search?tag=node-exporter-mixin&type=dash-db — Node Exporter dashboards
    • /api/search?tag=weatherapp(custom)&type=dash-db — custom weatherapp dashboards

    Deduplicate results by dashboard UID. For each discovered dashboard in the Azure Managed Prometheus folder (folderUid: azure-managed-prometheus), navigate to it with the correct datasource and cluster variables: var-datasource=ci-dev-aks-eus-mac&var-cluster=ci-dev-aks-mac-eus&from=<deployment-aware-range>. The datasource UID ci-dev-aks-eus-mac corresponds to Managed_Prometheus_ci-dev-aks-eus-mac which points to the ci-dev AMW endpoint. Wait for panels to load, and check for "No data" panels. Use Playwright's page.locator('text="No data"').count() to efficiently detect empty panels. Report a table of all dashboards grouped by tag with their total panel count and "No data" panel count. "No data" on error-rate, throttling, or swap I/O panels is expected when the system is healthy. If Playwright MCP is unavailable or auth fails, fall back to informing the user to verify manually.

Phase 3: Summary and Verdict

  1. Generate a Validation Summary Report using the template below. Fill in every row with actual results and the evidence that led to your pass/fail determination. Do NOT leave any row blank.
  2. Declare verdict: READY or NOT READY, with justification for any failures o

Content truncated.

When not to use it

  • →When the task is not related to validating a prometheus-collector release image
  • →When the task does not involve CI dev clusters or readiness for production
  • →When skipping any phase or step of the validation process

Limitations

  • →The skill is specific to validating a prometheus-collector release image
  • →The skill targets pipeline definition 440 and project `azure` in `github-private.visualstudio.com`
  • →The skill requires specific credentials for the `ci-dev-aks-mac-eus` cluster

How it compares

This skill provides a mandatory, multi-phase validation workflow for release-ready images, combining automated CI checks with detailed manual cluster inspections, offering a more thorough verification than relying solely on automated pipeli

Compared to similar skills

validate-release-ready-image-cidev side by side with the closest alternatives in the catalog.

SkillInstallsUpdatedSafetyDifficulty
validate-release-ready-image-cidev (this skill)06moReviewAdvanced
sentry-ci-integration14moReviewIntermediate
cloud-functions15moReviewIntermediate
genkit-infra-expert14moReviewAdvanced

Try saying

Example prompts that trigger this skill in your AI assistant.

You might also like

sentry-ci-integration

jeremylongshore

Manage integrate Sentry with CI/CD pipelines. Use when setting up GitHub Actions, GitLab CI, or other CI systems with Sentry releases and source maps. Trigger with phrases like "sentry github actions", "sentry CI", "sentry pipeline", "automate sentry releases".

113

cloud-functions

TencentCloudBase

Complete guide for CloudBase cloud functions development - runtime selection, deployment, logging, invocation, and HTTP access configuration.

18

genkit-infra-expert

jeremylongshore

Execute use when deploying Genkit applications to production with Terraform. Trigger with phrases like "deploy genkit terraform", "provision genkit infrastructure", "firebase functions terraform", "cloud run deployment", or "genkit production infrastructure". Provisions Firebase Functions, Cloud Run services, GKE clusters, monitoring dashboards, and CI/CD for AI workflows.

15

deploy-preflight

vincentxuu

Pre-deploy verification for the quidproquo Cloudflare Workers site — run lint / astro check / tests / check:references, sanity-check git state and wrangler config, and produce a go/no-go report before `pnpm deploy`. Does NOT deploy. Use when user says 準備 deploy / 上線前檢查 / preflight / 部署前看一下.

00

openclaw-4656-a-b-rollout-governance

zwright8

A/B Rollout Governance for product management execution. Use when work requires a/b rollout governance for product management execution with guardrails, traceable execution, and measurable outcomes.

00

gcp-cloud-run

aj-geddes

Deploy containerized applications on Google Cloud Run with automatic scaling, traffic management, and service mesh integration. Use for container-based serverless computing.

5107

Search skills

Search the agent skills registry