dp-solver
Solves small-scale tabular Markov Decision Processes using dynamic programming methods.
Install
mkdir -p .claude/skills/dp-solver && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/16967" && unzip -o skill.zip -d .claude/skills/dp-solver && rm skill.zipInstalls to .claude/skills/dp-solver
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Solve a small tabular MDP exactly via policy iteration or value iteration. Report convergence behavior. Use when you need help with dp solver.Key capabilities
- →Choose between Policy Iteration and Value Iteration
- →Initialize V_0 and starting policy for convergence sensitivity
- →Define stopping criteria using sup-norm tolerance ε
- →Verify V*(s_0) computation and greedy policy extraction
- →Explain how the baseline will debug/evaluate sampling-based methods
- →Report convergence behavior of the chosen algorithm
How it works
The skill guides the exact solution of a small tabular MDP by selecting either policy iteration or value iteration, defining initialization and stopping criteria, and verifying the computed optimal value function and policy.
Inputs & outputs
When to use dp-solver
- →Solve small MDPs
- →Compute optimal policy
- →Debug RL algorithms
- →Analyze convergence behavior
About this skill
Given an MDP with a known model, output:
- Choice. Policy iteration vs value iteration. Reason tied to |S|, |A|, γ.
- Initialization. V_0, starting policy. Convergence sensitivity.
- Stopping. Sup-norm tolerance ε. Expected number of sweeps.
- Verification. V*(s_0) computed exactly. Greedy policy extracted.
- Use. How this baseline will be used to debug/evaluate sampling-based methods.
Refuse to run DP on state spaces > 10⁷. Refuse to claim convergence without a sup-norm check. Flag any γ ≥ 1 on an infinite-horizon task as a guarantee violation.
When not to use it
- →When the state space is greater than 10⁷
- →When claiming convergence without a sup-norm check
- →When the discount factor γ is ≥ 1 for an infinite-horizon task
Limitations
- →Refuse to run DP on state spaces > 10⁷
- →Refuse to claim convergence without a sup-norm check
- →Flag any γ ≥ 1 on an infinite-horizon task as a guarantee violation
How it compares
This skill provides a structured approach to solving small tabular MDPs exactly, focusing on convergence behavior and verification, which serves as a baseline for debugging sampling-based reinforcement learning methods.
Compared to similar skills
dp-solver side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| dp-solver (this skill) | 0 | 2mo | No flags | Advanced |
| llava | 7 | 9mo | Review | Advanced |
| cocoindex | 6 | 10mo | Review | Intermediate |
| ai-multimodal | 9 | 7mo | Review | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
More by anubhavg-icpl
View all by anubhavg-icpl →You might also like
llava
zechenzhangAGI
Large Language and Vision Assistant. Enables visual instruction tuning and image-based conversations. Combines CLIP vision encoder with Vicuna/LLaMA language models. Supports multi-turn image chat, visual question answering, and instruction following. Use for vision-language chatbots or image understanding tasks. Best for conversational image analysis.
cocoindex
cocoindex-io
Comprehensive toolkit for developing with the CocoIndex library. Use when users need to create data transformation pipelines (flows), write custom functions, or operate flows via CLI or API. Covers building ETL workflows for AI data processing, including embedding documents into vector databases, building knowledge graphs, creating search indexes, or processing data streams with incremental updates.
ai-multimodal
mrgoonie
Process and generate multimedia content using Google Gemini API. Capabilities include analyze audio files (transcription with timestamps, summarization, speech understanding, music/sound analysis up to 9.5 hours), understand images (captioning, object detection, OCR, visual Q&A, segmentation), process videos (scene detection, Q&A, temporal analysis, YouTube URLs, up to 6 hours), extract from documents (PDF tables, forms, charts, diagrams, multi-page), generate images (text-to-image, editing, composition, refinement). Use when working with audio/video files, analyzing images or screenshots, processing PDF documents, extracting structured data from media, creating images from text prompts, or implementing multimodal AI features. Supports multiple models (Gemini 2.5/2.0) with context windows up to 2M tokens.
rag-implementation
wshobson
Build Retrieval-Augmented Generation (RAG) systems for LLM applications with vector databases and semantic search. Use when implementing knowledge-grounded AI, building document Q&A systems, or integrating LLMs with external knowledge bases.
rdkit
K-Dense-AI
Cheminformatics toolkit for fine-grained molecular control. SMILES/SDF parsing, descriptors (MW, LogP, TPSA), fingerprints, substructure search, 2D/3D generation, similarity, reactions. For standard workflows with simpler interface, use datamol (wrapper around RDKit). Use rdkit for advanced control, custom sanitization, specialized algorithms.
pyhealth
davila7
Comprehensive healthcare AI toolkit for developing, testing, and deploying machine learning models with clinical data. This skill should be used when working with electronic health records (EHR), clinical prediction tasks (mortality, readmission, drug recommendation), medical coding systems (ICD, NDC, ATC), physiological signals (EEG, ECG), healthcare datasets (MIMIC-III/IV, eICU, OMOP), or implementing deep learning models for healthcare applications (RETAIN, SafeDrug, Transformer, GNN).