PufferLib provides a framework for fast, scalable reinforcement learning and multi-agent training.

Install

mkdir -p .claude/skills/pufferlib && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/5176" && unzip -o skill.zip -d .claude/skills/pufferlib && rm skill.zip

Installs to .claude/skills/pufferlib

Activation

This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.

High-performance reinforcement learning framework optimized for speed and scale. Use when you need fast parallel training, vectorized environments, multi-agent systems, or integration with game environments (Atari, Procgen, NetHack). Achieves 2-10x speedups over standard implementations. For quick prototyping or standard algorithm implementations with extensive documentation, use stable-baselines3 instead.
409 chars✓ has a “when” triggerlonger than Claude Code's old 250-char listing cap (fine on current versions)
Advanced

Key capabilities

  • →Parallel environment simulation
  • →Vectorized environment training
  • →Multi-agent system support
  • →PPO algorithm implementation
  • →Integration with Gymnasium and PettingZoo

How it works

It utilizes optimized vectorization and shared memory buffers to achieve high-speed parallel training across multiple environments.

Inputs & outputs

You give it
Environment name or custom class
You get back
Trained policy or high-throughput simulation

When to use pufferlib

  • →Implementing multi-agent reinforcement learning
  • →Scaling training across parallel environments
  • →Integrating game environments for training

About this skill

PufferLib

Choose the version before choosing an API. Reviewed 2026-10-01:

ProfileStatusMain use
Native source 5.0Current default branch and live documentationC/CUDA environments, native trainer; CPU evaluation only
pufferlib==3.0.0Latest PyPI release, published 2025-06-23; sdist onlyPython/Gymnasium/PettingZoo adaptation and Torch PuffeRL
Pinned source 4.0Historical snapshotC Ocean interface with an optional Torch fallback

For 5.0, read references/native-5.md. The reviewed revision is 6ffa5b10dbbbe4d1e8288367c7d9d3acd3bad4a2. Its CLI is ./puffer train after building an environment, not puffer train ENV_NAME. There is no 5.0 Python emulation/vector API or --slowly fallback.

The bundled plan schema deliberately supports only 3.0 and pinned 4.0; it does not launch training. All native/PufferLib training examples are source-reviewed, illustrative, and not executed in this review. Bundled synthetic checks are executed CPU tests, not evidence of PufferLib installation or learning quality. The PyPI sdist was hash-verified and its Python sources inspected; the moving 3.0 branch differs, including its load_policy and logger contracts.

Safe defaults

  1. Start with bundled synthetic, CPU-only, network-free tools.
  2. Do not import an arbitrary environment by dotted path. Bundled tools accept only allowlisted built-ins and slug identifiers.
  3. Do not install or execute an unreviewed environment package, native extension, ROM, map, checkpoint, or pickle file.
  4. Verify official source, immutable revision, licenses, checksums or attestations, and build hooks. Sandbox native builds and first execution.
  5. Cap steps, environments, agents, workers, threads, buffers, memory, disk, render size, and wall time.
  6. Keep training and evaluation environments/seeds separate.
  7. Default logging to local/none. External logging requires explicit opt-in, disclosure acknowledgment, and separate artifact-upload approval.
  8. Never pass W&B or Neptune credentials via CLI, INI, JSON, tags, run names, or logger configuration. Never print them.
  9. Never dump all environment variables or recursively search for .env.
  10. Hash checkpoint bytes before trusted, sandboxed loading; metadata inspection is not proof of safety.

First local checks

All bundled CLIs are dependency-free and emit strict JSON:

python3 scripts/env_template.py --help
python3 scripts/env_contract_validator.py
python3 scripts/benchmark_vectorization.py --backend serial
python3 scripts/train_template.py
python3 scripts/validate_plan.py
python3 scripts/repro_plan.py

Defaults are synthetic, deterministic, bounded, local, CPU-only, no-network, and dry-run where training would otherwise occur.

Installation and provenance

Published 3.0.0

PyPI supplies only pufferlib-3.0.0.tar.gz:

sha256: 7df3a3e3f5f894d78d2a1f5374097890aec01473183e748abefe4f3faa10eaa9
Requires-Python: >=3.9

After source/build review, create a pinned uv project:

uv venv --python 3.11
uv add --exact --no-sync "pufferlib==3.0.0"
uv lock
uv sync --frozen

These installation commands are illustrative and were not executed. Commit pyproject.toml and uv.lock; verify the archive digest and every resolved dependency. The source build can compile native code and fetch build assets, so resolve/build in a sandbox without credentials or sensitive mounts. The archive declares NumPy <2, Gymnasium <=0.29.1, and PettingZoo <=1.24.1; latest Gymnasium/NumPy are not valid substitutes for this profile. Its setup supports Linux/macOS and rejects other systems. Python classifiers alone do not establish a successful native build. The uploaded metadata does not pin Torch or CUDA; do not claim a supported CUDA matrix that PyPI does not declare.

Pinned 4.0 source

The reviewed branch head on 2026-07-23 was:

25647630e1b15330bb3153a5a0d3ff8d234c3acf

Pin the commit, not branch 4.0:

uv add --no-sync \
  "pufferlib @ git+https://github.com/PufferAI/PufferLib.git@25647630e1b15330bb3153a5a0d3ff8d234c3acf"
uv lock

The reviewed 4.0 package declares Python >=3.10 and Torch >=2.9. The reviewed PufferTank snapshot uses Ubuntu 24.04, Python 3.12, and an NVIDIA CUDA 13.0.2/cuDNN development image with the cu130 Torch index, but does not pin the exact Torch wheel or all system packages. Treat it as a reference, not a complete lock. Never execute a remote installer directly from a pipe.

Read references/training.md before any installation or build.

Environment workflow

1. Validate the contract

Gymnasium reset returns (observation, info). Step returns:

(observation, reward, terminated, truncated, info)

Validate spaces, shapes, dtypes, finite rewards, booleans, reset-before-step, reset-after-end, seeding, and cleanup. terminated is an MDP terminal; truncated is an external cutoff such as a time limit. Preserve the distinction for bootstrapping and metrics. Both flags can be true in general Gymnasium. Check autoreset timing and retain the final pre-reset observation; do not bootstrap from the next episode. The 3.0 trainer has unresolved truncation and inactive-agent mask handling, described in references/training.md.

python3 scripts/env_contract_validator.py \
  --steps 64 --episodes 8 --seed 42

2. Adapt only after review

Published 3.0 uses explicit wrappers:

import pufferlib.emulation

wrapped = pufferlib.emulation.GymnasiumPufferEnv(reviewed_gymnasium_instance)

For a reviewed PettingZoo Parallel environment:

wrapped = pufferlib.emulation.PettingZooPufferEnv(reviewed_parallel_instance)

There is no supported 3.0 pufferlib.emulate(...) shortcut matching the old skill. Read references/environments.md and references/integration.md.

3. Native environments

Published 3.0 PufferEnv requires single_observation_space, single_action_space, and num_agents before super().__init__(buf). It uses in-place vector buffers and returns separate terminal/truncation arrays plus a list of info dictionaries.

The reviewed 4.0 source uses C bindings. Start from upstream ocean/squared (single-agent) or ocean/target (multi-agent), build one environment in local/sanitized mode, and verify every buffer size/type/index before optimization.

Vectorization workflow

Published 3.0:

import pufferlib.vector

vecenv = pufferlib.vector.make(
    reviewed_creator,
    backend=pufferlib.vector.Serial,
    num_envs=4,
    seed=42,
)

Move to Multiprocessing only after serial traces pass. Record num_envs, num_workers, batch_size, zero-copy mode, start method, agent count, masks, and actual returned shapes. For multi-agent environments, batch length is based on agent slots, not necessarily num_envs.

The reviewed 4.0 config instead uses:

[vec]
total_agents = 4096
num_buffers = 2
num_threads = 16

Read references/vectorization.md. Benchmark fixed work with warmup and at least three repeats; report simulation and end-to-end training SPS separately. The bundled benchmark measures only its synthetic harness.

Policy workflow

Published 3.0 policies are Torch modules sized from single_observation_space/single_action_space. Stable recurrent composition uses encode_observations and decode_actions; structured emulation uses pufferlib.pytorch.nativize_dtype and nativize_tensor.

The reviewed 4.0 Torch fallback composes:

pufferlib.models.Policy(encoder=encoder, decoder=decoder, network=network)

It provides MLP, MinGRU, LSTM, and GRU network choices; --slowly selects this fallback instead of the native backend. Check output/state shapes, masks, finite values, gradients, and eager-versus-compiled behavior. See references/policies.md.

Training and evaluation

Published 3.0 trainer import:

from pufferlib import pufferl

# train_config must include the environment name for checkpoint naming.
trainer = pufferl.PuffeRL(train_config, vecenv, policy)

Reviewed 4.0 CLI:

puffer train ENV_NAME
puffer eval ENV_NAME --load-model-path EXACT_TRUSTED_PATH
puffer sweep ENV_NAME

Generate a plan instead of launching by default:

python3 scripts/train_template.py \
  --profile pypi-3.0.0 \
  --environment synthetic \
  --device cpu \
  --total-timesteps 10000

train_template.py emits a report envelope, not a bare plan. Its plan member is the input to validate_plan.py; passing the whole report is invalid. For the synthetic environment, command_preview is an empty list because there is no upstream training command to launch. To save and revalidate:

python3 scripts/train_template.py > training-report.json
python3 -c 'import json; r=json.load(open("training-report.json")); print(json.dumps(r["plan"], allow_nan=False, indent=2))' > plan.json
python3 scripts/validate_plan.py --root . --config plan.json

The handoff consists of the training report, extracted plan and validation report. A command preview exists only for a reviewed non-synthetic environment and remains partial until its environment-specific settings are resolved.

Validate a custom strict-JSON plan using the same bare-plan format:

python3 scripts/validate_plan.py --root . --config plan.json

The schema rejects secret-bearing keys, unbounded resources, dotted environment paths, invalid vector divisibility, mixed-version options, and coupled train/eval seeds. See references/training.md.

Logging

PufferLib 3.0 contains historical W&B and Neptune integrations; pinned 4.0 contains W&B. Neptune shut down on 2026-03-05 and the bundled planner rejects it. Native 5.0 uses local logs/Constellation and has no reviewed W&B or Neptune CLI flag. W&B remains an optional external service. It may transmit configuration, metrics, source metadata, hardware telemetry, outp


Content truncated.

When not to use it

  • →For quick prototyping of standard algorithms
  • →When extensive documentation is the primary requirement

Prerequisites

PyTorch

Limitations

  • →Higher complexity than stable-baselines3
  • →Requires careful environment vectorization

How it compares

It is specifically optimized for speed and scale, offering 2-10x performance gains over standard RL implementations.

Compared to similar skills

pufferlib side by side with the closest alternatives in the catalog.

SkillInstallsUpdatedSafetyDifficulty
pufferlib (this skill)15moReviewAdvanced
llama-cpp211yReviewIntermediate
langchain261yReviewIntermediate
unsloth151yNo flagsIntermediate

Try saying

Example prompts that trigger this skill in your AI assistant.

More by K-Dense-AI

View all by K-Dense-AI →

literature-review

K-Dense-AI

Conduct comprehensive, systematic literature reviews using multiple academic databases (PubMed, arXiv, bioRxiv, Semantic Scholar, etc.). This skill should be used when conducting systematic literature reviews, meta-analyses, research synthesis, or comprehensive literature searches across biomedical, scientific, and technical domains. Creates professionally formatted markdown documents and PDFs with verified citations in multiple citation styles (APA, Nature, Vancouver, etc.).

5591,298

markitdown

K-Dense-AI

Convert various file formats (PDF, Office documents, images, audio, web content, structured data) to Markdown optimized for LLM processing. Use when converting documents to markdown, extracting text from PDFs/Office files, transcribing audio, performing OCR on images, extracting YouTube transcripts, or processing batches of files. Supports 20+ formats including DOCX, XLSX, PPTX, PDF, HTML, EPUB, CSV, JSON, images with OCR, and audio with transcription.

177310

scientific-writing

K-Dense-AI

Write scientific manuscripts. IMRAD structure, citations (APA/AMA/Vancouver), figures/tables, reporting guidelines (CONSORT/STROBE/PRISMA), abstracts, for research papers and journal submissions.

94309

exploratory-data-analysis

K-Dense-AI

Perform comprehensive exploratory data analysis on scientific data files across 200+ file formats. This skill should be used when analyzing any scientific data file to understand its structure, content, quality, and characteristics. Automatically detects file type and generates detailed markdown reports with format-specific analysis, quality metrics, and downstream analysis recommendations. Covers chemistry, bioinformatics, microscopy, spectroscopy, proteomics, metabolomics, and general scientific data formats.

15114

infographics

K-Dense-AI

Create professional infographics using Nano Banana Pro AI with smart iterative refinement. Uses Gemini 3 Pro for quality review. Integrates research-lookup and web search for accurate data. Supports 10 infographic types, 8 industry styles, and colorblind-safe palettes.

1141

pptx-posters

K-Dense-AI

Create research posters using HTML/CSS that can be exported to PDF or PPTX. Use this skill ONLY when the user explicitly requests PowerPoint/PPTX poster format. For standard research posters, use latex-posters instead. This skill provides modern web-based poster design with responsive layouts and easy visual integration.

911

You might also like

llama-cpp

zechenzhangAGI

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

21471

langchain

zechenzhangAGI

Framework for building LLM-powered applications with agents, chains, and RAG. Supports multiple providers (OpenAI, Anthropic, Google), 500+ integrations, ReAct agents, tool calling, memory management, and vector store retrieval. Use for building chatbots, question-answering systems, autonomous agents, or RAG applications. Best for rapid prototyping and production deployments.

26138

unsloth

zechenzhangAGI

Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization

15117

llama-factory

zechenzhangAGI

Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support

15112

llava

zechenzhangAGI

Large Language and Vision Assistant. Enables visual instruction tuning and image-based conversations. Combines CLIP vision encoder with Vicuna/LLaMA language models. Supports multi-turn image chat, visual question answering, and instruction following. Use for vision-language chatbots or image understanding tasks. Best for conversational image analysis.

7117

cocoindex

cocoindex-io

Comprehensive toolkit for developing with the CocoIndex library. Use when users need to create data transformation pipelines (flows), write custom functions, or operate flows via CLI or API. Covers building ETL workflows for AI data processing, including embedding documents into vector databases, building knowledge graphs, creating search indexes, or processing data streams with incremental updates.

6116

Search skills

Search the agent skills registry