vastai-reference-architecture

Governed Vast.ai GPU Workload Architecture

Overview

Center the architecture on an immutable run or release manifest and a lifecycle ledger. Search and planning are read-only; paid resource creation crosses an approval boundary; execution writes recoverable state externally; teardown closes both cost and evidence.

Prerequisites

• Batch, training, interactive, or Serverless workload inventory with SLOs
• Data, model, image, credential, region, reliability, and spend policies
• Owners for approval, execution, recovery, billing, security, and platform incidents

Instructions

Step 1: Define the immutable intent

Create a signed or versioned manifest containing workload bytes, image/template/model identity, GPU policy, data/checkpoint routes, SLOs, budget, and expiry.

Step 2: Separate planner and mutator

Let a read-scoped planner evaluate offers or Serverless profiles. Require explicit approval before a narrowly scoped mutator creates, updates, transfers credit, or destroys.

Step 3: Choose the executor

Use an instance lifecycle for bounded jobs or dedicated services; use Serverless endpoint/workergroup control for managed inference scaling and rolling updates.

Step 4: Externalize durable state

Keep datasets, checkpoints, artifacts, event ledgers, and evidence outside disposable root disks with checksums and recovery objectives.

Step 5: Observe and reconcile

Combine provider states, signed notifications, bounded polling, workload SLOs, balance, charges, and resource inventory; reconcile events against periodic reads.

Step 6: Close every lifecycle

Accept output, copy evidence, destroy disposable resources, revoke temporary access, reconcile charges, and leave an auditable handoff for retained resources.

Authentication

Use native Teams roles and distinct scoped keys for planning, mutation, monitoring, and administration. Workload storage and registry credentials must never inherit control-plane authority.

Tool Discipline

Use Read and Grep to inspect manifests, configuration, provider output, and existing tests before proposing a mutation. Use Write or Edit only for the approved plan, implementation, test, or redacted receipt; do not create, update, destroy, or fund Vast.ai resources without explicit operator approval.

Output

• Trust-boundary and component decision record
• Immutable manifest, lifecycle ledger, recovery, and observability contracts
• Threat, failure, cost, rollback, and teardown evidence plan

Return workload classes, chosen executors, authority boundaries, immutable artifacts, recovery targets, SLOs, budgets, event reconciliation, and lifecycle owners.

Examples

A planner selects verified offers but cannot rent; an approved mutator creates from a signed run manifest; the training executor checkpoints externally; a signed event plus reconciliation loop detects failure; a finalizer destroys the instance and closes the charge ledger.

Error Handling

Resources

• First-party source notes
• Vast.ai concepts
• Serverless architecture
• API permissions