coderabbit-observability

CodeRabbit Observability

Overview

Use documented metric definitions and preserve denominators. Avoid scraping bot comments as a substitute for authoritative reporting.

Prerequisites

• Identify the CodeRabbit organization, Git provider, repository, plan, and accountable owner.
• Read references/official-docs.md and re-check any time-sensitive contract before execution.
• Use synthetic or read-only evidence until the approval boundary is satisfied.
• Preserve the repository's independent CI, security, and human-review requirements.

Current Contract

• Git-platform and IDE or CLI metrics have separate definitions.
• The Usage dashboard explains events, limits, and the Review Log.
• Data export supports offline analysis.
• Authenticated metrics APIs depend on plan and permissions.

Authentication

Treat Git-provider sessions, CodeRabbit web sessions, CLI credentials, and CodeRabbit API keys as separate credentials. Use only an already-approved session or secret-manager reference, never print a secret, and do not place credentials in .coderabbit.yaml, source files, logs, or deliverables.

Instructions

1. Define decision, population, provider, repositories, UTC window, and metrics.

2. Export with timestamps, filters, pagination, and exclusions.

3. Compute coverage, response, resolution, and usage with denominators.

4. Compare cohorts and publish thresholds only after baseline evidence.

Tool Discipline

• Use Glob to locate candidate configuration and evidence files without widening scope.
• Use Grep to find relevant fields, commands, identifiers, and stale claims.
• Use Read to inspect the smallest required files and authoritative evidence.
• Use Write only for a new approved local draft or evidence artifact.
• Use Edit only for a bounded approved change whose rollback is known.
• Do not use these file tools as a substitute for authenticated CodeRabbit or provider operations.

Approval Boundaries

Require data-owner approval before user-level exports, scheduled reports, or API credentials. Keep analysis and drafts local until approval is explicit, and record who approved the action and its scope.

Output

A reproducible metric spec, source receipt, dataset summary, dashboard, anomalies, and actions. Include source dates, unknowns, and the exact boundary between observed fact and recommendation.

Error Handling

Examples

Example 1

Compare coverage before and after central config rollout.

Example 2

Reconcile CLI use with Review Log before buying credits.

Validation

• Confirm every claim against the dated sources in references/official-docs.md.
• Verify the requested scope, owner, approval, happy path, failure path, and rollback.
• Re-read the effective configuration or provider state after any approved change.
• Report unsupported fields, undocumented endpoints, and unverified assumptions as failures.

Resources

• Official documentation and contract notes
• Re-check the dated contract before any live operation.
• Treat unresolved or changed vendor behavior as a stop condition.