vastai-observability
Implement comprehensive observability for Vast.ai to track GPU health, utilization, and costs.
Install
mkdir -p .claude/skills/vastai-observability && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/2394" && unzip -o skill.zip -d .claude/skills/vastai-observability && rm skill.zipInstalls to .claude/skills/vastai-observability
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Monitor Vast.ai GPU instance health, utilization, and costs.Key capabilities
- →Collect Vast.ai instance metrics like GPU utilization and cost
- →Generate alerts for idle GPUs and high temperatures
- →Project daily cost based on running instances
- →Monitor remote GPU metrics via SSH and `nvidia-smi`
- →Export metrics to Prometheus for dashboarding
How it works
The skill collects Vast.ai instance data using the `vastai` CLI, processes it with Python to extract metrics, and generates alerts based on predefined thresholds. It can also export these metrics for visualization tools.
Inputs & outputs
When to use vastai-observability
- →Set up custom monitoring dashboards
- →Configure alerts for GPU preemption
- →Track real-time hourly spend per instance
- →Monitor GPU utilization to identify wasted compute
About this skill
Vast.ai Control-Plane Observability
Overview
Measure provider state and workload health separately. Alerts must identify a resource, threshold, evidence link, responder, and safe action; a dashboard without terminal-state and billing coverage is incomplete.
Prerequisites
- Instance, endpoint, workergroup, deployment, and account scopes
- Latency, error, queue, utilization, state-age, balance, and cost objectives
- Collection interval, retention, alert routing, and incident owner
Instructions
Step 1: Inventory labeled resources
Map instance labels and Serverless IDs to service, environment, release, cost center, and owner.
Step 2: Collect structured control state
Read instance actual status, timestamps, price, disk, and endpoints; collect endpoint/workergroup status, logs, and deployment versions.
Step 3: Collect workload signals
Measure request/job success, latency, queue time, GPU utilization, memory, disk, checkpoint age, and last successful artifact.
Step 4: Add billing protection
Track credit balance, instance and storage charges, active/stopped age, and orphaned resources. Alert before balance or cleanup risk becomes urgent.
Step 5: Define stateful alerts
Alert on terminal states, excessive transition age, queue/SLO breach, checkpoint staleness, low balance, and cleanup failure with deduplication.
Step 6: Test the path
Inject a canary event or threshold breach, verify delivery and ownership, then record recovery and false-positive behavior.
Authentication
Use read-only keys for collectors and distinct secrets for alert sinks. Never put a mutation-capable Vast.ai key in dashboards or telemetry processors.
Tool Discipline
Use Read and Grep to inspect manifests, configuration, provider output, and existing tests before proposing a mutation. Use Write or Edit only for the approved plan, implementation, test, or redacted receipt; do not create, update, destroy, or fund Vast.ai resources without explicit operator approval.
Output
- Resource-to-owner inventory and telemetry schema
- Dashboard and actionable alert definitions
- Alert-path test, retention, and unresolved coverage receipt
Return resource scope, collection interval, SLOs, tested alert, responder, evidence location, and blind spots.
Examples
A dashboard separates an instance's running state from workload request success, pages on stale external checkpoints and low balance, and assigns stopped-storage leaks to the billing owner.
Error Handling
| Failure | Response |
|---|---|
| Collector receives 403 | Add only the documented read category needed by that metric. |
| Resource is missing from inventory | Quarantine the alert and assign ownership before automated action. |
| Metrics lag exceeds the SLO | Mark the dashboard stale and use direct provider state during incidents. |
| Alert contains a secret or payload | Disable the route, scrub data, rotate credentials, and narrow fields. |
Resources
When not to use it
- →When an instance is offline and status changed from running
Prerequisites
Limitations
- →Idle GPU alert triggers if utilization is less than 10% for more than 10 minutes
- →High temperature alert triggers if GPU temperature exceeds 85C
- →Budget alert triggers if projected daily cost exceeds $100
How it compares
This skill provides automated monitoring and alerting for Vast.ai GPU instances, offering real-time insights into utilization and costs, unlike manual checks that can miss critical events.
Compared to similar skills
vastai-observability side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| vastai-observability (this skill) | 1 | 2mo | Review | Intermediate |
| distributed-tracing | 5 | 4mo | No flags | Intermediate |
| service-mesh-observability | 5 | 4mo | No flags | Advanced |
| observability-engineer | 12 | 5mo | No flags | Advanced |
Try saying
Example prompts that trigger this skill in your AI assistant.
More by jeremylongshore
View all by jeremylongshore →You might also like
distributed-tracing
wshobson
Implement distributed tracing with Jaeger and Tempo to track requests across microservices and identify performance bottlenecks. Use when debugging microservices, analyzing request flows, or implementing observability for distributed systems.
service-mesh-observability
wshobson
Implement comprehensive observability for service meshes including distributed tracing, metrics, and visualization. Use when setting up mesh monitoring, debugging latency issues, or implementing SLOs for service communication.
observability-engineer
sickn33
Build production-ready monitoring, logging, and tracing systems. Implements comprehensive observability strategies, SLI/SLO management, and incident response workflows. Use PROACTIVELY for monitoring infrastructure, performance optimization, or production reliability.
prometheus-configuration
wshobson
Set up Prometheus for comprehensive metric collection, storage, and monitoring of infrastructure and applications. Use when implementing metrics collection, setting up monitoring infrastructure, or configuring alerting systems.
langfuse
davila7
Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.
slo-implementation
wshobson
Define and implement Service Level Indicators (SLIs) and Service Level Objectives (SLOs) with error budgets and alerting. Use when establishing reliability targets, implementing SRE practices, or measuring service performance.