agentskills.codes
GR

grpo-debug

by Jackie2049CodingSource

Diagnostic tool for fixing GRPO training failures in complex machine learning models.

DebuggingMachine Learning

Install

mkdir -p .claude/skills/grpo-debug && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/19146" && unzip -o skill.zip -d .claude/skills/grpo-debug && rm skill.zip

Installs to .claude/skills/grpo-debug

Activation

This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.

Debug and diagnose GRPO training issues across verl, Megatron, DeepSpeed, and RTX 4090. Symptom-to-bug mapping, NaN debugging, Muon clipping avoidance, and CUDA stream safety analysis.
184 charsno explicit “when” trigger
Advanced

Key capabilities

  • →Identify NaN in loss during GRPO training
  • →Diagnose loss plateau issues in GRPO training
  • →Pinpoint gradient norm explosion problems
  • →Analyze CUDA stream safety for training configurations
  • →Avoid Muon clipping stalls in Megatron
  • →Debug specific GRPO bugs across frameworks like DeepSpeed and Megatron

How it works

The skill maps GRPO training symptoms to known bug patterns across various frameworks and provides diagnostic tools and rules to identify root causes and suggest fixes.

Inputs & outputs

You give it
GRPO training symptoms (e.g., NaN in loss, loss plateau, gradient explosion)
You get back
Identification of the most likely bug and recommended fix

When to use grpo-debug

  • →Debugging training NaN
  • →Fixing gradient explosion
  • →Analyzing CUDA stream safety

About this skill

GRPO Debug & Diagnostic Skill

When to Use

  • User encounters NaN, loss plateau, gradient explosion during GRPO training
  • User asks about debugging GRPO training failures
  • User wants to diagnose Muon clipping or ChainedOptimizer issues
  • User needs CUDA stream safety analysis for training
  • User asks about verl/Megatron/DeepSpeed GRPO bugs
  • User wants symptom-to-bug mapping for training issues

7 Pattern Classes × Bug Mapping

Pattern ClassBugsFrameworksRoot Cause
cuda_stream_safety#8061, #45552, #28499DeepSpeed, vLLM, PyTorchrecord_stream missing or wrong stream
muon_clipping#8068, #7776, #5394, #5395DeepSpeed, Megatron, verlScale-invariant optimizer + global clipping = stall
discrete_decision_mismatch#5384, #5386Megatron, verlTrain/rollout top-k mismatch → wrong GRPO signal
lora_distortion#6782, #28566vLLM, SGLangLoRA rank>=64 breaks EOS / cache corruption
execution_overlap_oom#6772, #45552PyTorch FSDP2, vLLMOverlap all_gather + backward → OOM
singleton_degeneration#605, #663rLLM, verlgs=1 = REINFORCE(baseline=0)
moe_fp16_nan#10579, Switch TFvLLM-Ascend, CUDAFP16 softmax overflow → NaN

Symptom → Bug Quick Map

SymptomMost Likely BugFix
NaN in loss#8061 overlap_comm+torch.compile or #10579 FP16 softmaxoverlap_comm=False or FP32 gating softmax
Loss plateau (no improvement)#5394/#5395 Muon clipping stallSwitch to Adam, remove ChainedOptimizer global clip
Gradient norm → infinity#8068 gradient_clipping=0.0Set gradient_clipping=1.0 explicitly
All rewards = 0.0#663 Step.output null or #605 gs=1 degenerationFix null handling, use gs>=4
logprob mismatch train vs rollout#5384/#5386 DSA indexer replayImplement DSAIndexerReplay
EOS token missing in output#6782 LoRA rank=64 distortionUse LoRA rank=32
OOM on FSDP2#6772 execution overlapUse FSDP1, NOT FSDP2
MoE NaN (any platform)#10579 FP16 gating softmax overflowAlways compute gating softmax in FP32

Diagnostic Tools

1. GRPO Training Debug Playbook

python3 tools/grpo_training_debug_playbook.py playbook    # Full playbook with 7 pattern classes
python3 tools/grpo_training_debug_playbook.py symptoms     # Symptom→bug mapping
python3 tools/grpo_training_debug_playbook.py fixes        # Fix reference for each bug
python3 tools/grpo_training_debug_playbook.py rtx4090      # RTX 4090 specific debug report

2. GRPO NaN Debugging Guide

python3 tools/grpo_nan_debugging_guide.py check            # Check config for NaN risks
python3 tools/grpo_nan_debugging_guide.py diagnose         # Diagnose NaN from symptoms
python3 tools/grpo_nan_debugging_guide.py avoid            # Generate avoidance recommendations
python3 tools/grpo_nan_debugging_guide.py rtx4090          # RTX 4090 NaN risk analysis

3. Megatron Muon Clipping Avoidance

python3 tools/megatron_muon_clipping_avoidance_tool.py check       # Check Megatron config
python3 tools/megatron_muon_clipping_avoidance_tool.py diagnose    # Diagnose from symptoms
python3 tools/megatron_muon_clipping_avoidance_tool.py avoid       # Generate avoidance config
python3 tools/megatron_muon_clipping_avoidance_tool.py rtx4090     # RTX 4090 Muon analysis

4. CUDA Stream Safety Pattern Synthesis

python3 tools/cuda_stream_safety_pattern_synthesis.py map           # Map stream patterns across frameworks
python3 tools/cuda_stream_safety_pattern_synthesis.py analyze       # Analyze stream safety for a config
python3 tools/cuda_stream_safety_pattern_synthesis.py compare       # Compare frameworks
python3 tools/cuda_stream_safety_pattern_synthesis.py rtx4090       # RTX 4090 stream safety report

5. Algorithm Infra Knowledge Map (theory→bug→decision)

python3 tools/algorithm_infra_knowledge_map.py theory     # Theory derivation for a bug
python3 tools/algorithm_infra_knowledge_map.py bug         # Bug→pattern→decision chain
python3 tools/algorithm_infra_knowledge_map.py decision    # Decision→theory justification
python3 tools/algorithm_infra_knowledge_map.py rtx4090     # RTX 4090 knowledge report
python3 tools/algorithm_infra_knowledge_map.py proof       # Mathematical proof for a rule
python3 tools/algorithm_infra_knowledge_map.py cross       # Cross-framework pattern analysis

MUST DO Rules (GRPO Training)

  1. ZeRO-2 + CPU_Adam — Never ZeRO-3 on dp=1 (pure overhead)
  2. clip_grad=1.0 — Never leave default 0.0 (silently disabled → gradient explosion #8068)
  3. bypass_mode — Skip old_log_prob forward → 18Ψ→3.8Ψ memory savings
  4. group_by_prompt — Correct GRPO advantage normalization (not global)
  5. enforce_eager=True — Disable CUDA graph for training (safety)
  6. FSDP1 — Only proven safe for GRPO (FSDP2 BLOCKER: #6468 CPU leak + #6772 OOM)
  7. gs>=4 — Minimum group size (gs=1 = REINFORCE degeneration)
  8. LoRA r=32 + bypass sync — LoRA rank>=64 breaks EOS (#6782)
  9. naive checkpoint dp=1 — No distributed checkpoint overhead
  10. record_stream — Must call for async CUDA operations
  11. ulimit 65535 — File descriptor limit for NCCL
  12. shaped rewards — Use reward shaping, not flat outcome rewards
  13. gs>=8 for sparse — Sparse MoE needs larger groups
  14. sleep_level=1 — Never sleep_level=2 on RTX 4090 (weight corruption)
  15. SGLang rollout — SGLang > vLLM for GRPO (sleep/wake support)
  16. DSAIndexerReplay — Replay DSA indices for train/rollout consistency
  17. FP32 gating softmax — Always compute MoE gating in FP32 (#10579 universal)
  18. ECHO auxiliary loss — Free env-observation loss on verl (#668)

MUST NOT Rules (GRPO Training)

  1. ZeRO-3 — Pure overhead on dp=1, MUST use ZeRO-2
  2. Muon optimizer — Scale-invariant + clipping = stall (#5394/#5395/#8068)
  3. overlap_comm=True dp=1 — NaN with torch.compile (#8061) + no benefit
  4. sleep_level=2 RTX 4090 — Weight corruption risk
  5. LoRA rank>=64 vLLM — Breaks EOS (#6782)
  6. FSDP2 — BLOCKER: CPU leak + execution overlap OOM
  7. gs=1 — REINFORCE degeneration, MUST use gs>=4
  8. PPO-clip RTX 4090 — OOM (65.06 GiB > 24 GiB)
  9. full param sync — LoRA delta is sufficient for weight sync
  10. NCCL checkpoint dp=1 — Use naive checkpoint
  11. outcome reward gs<16 — Too noisy for small groups
  12. merge vLLM encoder cache revert — Known corruption bug
  13. global clipping Muon ChainedOptimizer — Fundamental contradiction
  14. FP16 gating softmax for MoE — Universal NaN (#10579)
  15. torch.compile + overlap_comm — NaN (#8061)
  16. gradient_clipping=0.0 — Silently disabled (#8068)
  17. bf16+fp16 both enabled — Choose bf16 only (SM89 native)

Deep Reading References

  • verl V1 GRPO training loop: notebook/projects/verl-v1-sync-ppo-trainer-grpo-training-loop-deep-reading.md
  • verl V1 critical bugs: notebook/projects/verl-v1-critical-bugs-issues-2026-06-20.md
  • DeepSpeed ZeRO gradient flow + stream safety: notebook/projects/deepspeed-zero1-2-gradient-flow-stream-safety-deep-reading.md
  • Megatron Muon clipping: notebook/projects/megatron-5394-chained-optimizer-muon-clipping-reading.md
  • GRPO singleton degeneration derivation: notebook/fundamentals/state-lifecycle-mismatch-pattern-family-derivation.md
  • RTX 4090 GRPO runbook: notebook/projects/rtx4090-grpo-training-runbook.md
  • Always load: notebook/auto-memory.md

RTX 4090 Specific Debug

Optimal GRPO config (16.24 GiB peak / 24 GiB VRAM):

trainer_mode=sync | rollout.n=8 | bypass_mode=True | loss_type=ppo_clip
actor.strategy=fsdp | lora_rank=32 | enforce_eager=True | checkpoint=naive
use_critic=False | ref_in_actor=True | gradient_clipping=1.0
sleep_level=1 | free_cache_engine=True | overlap_comm=False

If NaN appears on RTX 4090: Check overlap_comm (must be False), gradient_clipping (must be 1.0), gating dtype (must be FP32 for MoE), LoRA rank (must be 32).

If loss plateau on RTX 4090: Check optimizer (not Muon), clip_grad (not 0.0), group_size (>=4).

When not to use it

  • →When gradient_clipping is set to 0.0
  • →When overlap_comm=True and dp=1 with torch.compile
  • →When using FSDP2 for GRPO training

Limitations

  • →It focuses specifically on GRPO training issues
  • →It provides fixes for known bugs, not general training optimization
  • →It requires specific configurations to avoid certain issues

How it compares

This skill offers a structured symptom-to-bug mapping and specific diagnostic tools for GRPO training issues, which is more targeted than general debugging approaches.

Compared to similar skills

grpo-debug side by side with the closest alternatives in the catalog.

SkillInstallsUpdatedSafetyDifficulty
grpo-debug (this skill)013dReviewAdvanced
langsmith-observability46moReviewIntermediate
mflux-debugging31moNo flagsAdvanced
debug-cuda-crash26moReviewAdvanced

Try saying

Example prompts that trigger this skill in your AI assistant.

You might also like

langsmith-observability

davila7

LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.

430

mflux-debugging

filipstrand

Debug MLX ports by comparing against a PyTorch/diffusers reference via exported tensors/images (export-then-compare).

310

debug-cuda-crash

flashinfer-ai

Tutorial for debugging CUDA crashes using API logging

210

debug-distributed

inclusionAI

Guide for debugging distributed training issues in AReaL. Use when user encounters hangs, wrong results, OOM, or communication errors.

15

aoti-debug

pytorch

Debug AOTInductor (AOTI) errors and crashes. Use when encountering AOTI segfaults, device mismatch errors, constant loading failures, or runtime errors from aot_compile, aot_load, aoti_compile_and_package, or aoti_load_package.

14

reasoning-trace-optimizer

muratcankoylan

Debug and optimize AI agents by analyzing reasoning traces. Activates on 'debug agent', 'optimize prompt', 'analyze reasoning', 'why did the agent fail', 'improve agent performance', or when diagnosing agent failures and context degradation.

12

Search skills

Search the agent skills registry