check-prod

Check Production

Perform a read-only health assessment of resources in rg-ltc-dev. Discover resource names at runtime; do not rely on generated suffixes. Require an authenticated Azure CLI session.

Run independent Azure Monitor and Log Analytics queries concurrently after resource discovery. Cover:

• /ready status and response time
• Azure resource health and fired alerts (24h)
• availability failures, request count, 5xx count, and P95 latency (24h)
• exceptions (7d) and error-level traces (24h)
• dependency failures and latency, especially PostgreSQL and api.github.com
• PostgreSQL peak CPU, memory, storage, connections, and minimum CPU credits
• Container App peak CPU/memory and current-revision crash or unhealthy events
• console Traceback, FATAL, OOMKilled, or segmentation-fault events
• verification outcomes and OAuth success/failure activity (24h)
• API worker failures (verification.worker.failed), overdue attempts (verification.attempt.stuck), and queued versus executing backlog

Use workspace-mode tables AppRequests, AppExceptions, AppTraces, AppDependencies, AppAvailabilityResults, and AppMetrics. Container App tables may use either the _CL schema with _s columns or the standard schema.

Verdict

Critical: readiness is non-200; any 5xx; unavailable resource; PostgreSQL dependency failure; OOM/crash; fired Sev1 alert; DB CPU above 80% or credits below 10.

Warning: P95 above 500 ms; failed availability test; recurring exception; other dependency failure; DB CPU 50-80%, memory/storage 70-85%, or credits 10-30; Container App CPU/memory above 80%; unhealthy replicas without matching scale events; OAuth callback failures above 50% of observed OAuth callback outcomes when login activity exists.

Use auth.login.success and auth.callback.* failure events for OAuth outcomes. Expected request 401s for missing sessions and 303 login redirects are not OAuth callback failures or unhandled application errors. Request URL attributes should use route templates, including router prefixes; /unmatched is reserved for requests without a known route template.

Verification runs in the API under learn-to-cloud-api, not a separate host. Check worker failures in AppTraces as well as AppExceptions. Both /health and /ready return 503 when the worker task has finished; ordinary page routes are not worker health checks. The worker logs only error.type, not raw exception details. Overdue reasons are queued_beyond_limit and execution_beyond_limit, with verification.attempt.age_seconds. Correlate created, execution-started, and saved completion events by attempt ID; missing completion telemetry alone is not proof of pending work. Confirm backlog with authorized read-only database access. Treat worker death as Critical and overdue work as Warning. Do not log prompts, evidence, fetched source, or credentials while investigating.

Otherwise report Healthy. Missing telemetry is Unknown, not healthy.

Return one overall verdict followed by a compact table containing each signal, its status, observed value, and time window. Put actionable critical findings first and distinguish application failures from Azure telemetry-query failures.