querying-mlflow-metrics
Fetches and analyzes LLM trace metrics and quality evaluations from MLflow.
Install
mkdir -p .claude/skills/querying-mlflow-metrics && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/16599" && unzip -o skill.zip -d .claude/skills/querying-mlflow-metrics && rm skill.zipInstalls to .claude/skills/querying-mlflow-metrics
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Fetches aggregated trace metrics (token usage, latency, trace counts, quality evaluations) from MLflow tracking servers. Triggers on requests to show metrics, analyze token usage, view LLM costs, check usage trends, or query trace statistics.Key capabilities
- →Query MLflow tracking servers for trace metrics
- →Analyze token usage summaries and trends
- →Retrieve latency percentiles by trace name
- →Calculate error rates by trace status
- →Fetch quality scores by evaluator
- →Obtain assessment counts by name
How it works
This skill executes the `scripts/fetch_metrics.py` script to query MLflow tracking servers, allowing users to specify metrics, aggregations, dimensions, and time intervals. It retrieves and formats data related to token usage, latency, trace counts, and quality evaluations.
Inputs & outputs
When to use querying-mlflow-metrics
- →Analyzing token usage trends
- →Querying LLM operational costs
- →Checking latency percentiles by trace
- →Calculating average quality scores
About this skill
MLflow Metrics
Run scripts/fetch_metrics.py to query metrics from an MLflow tracking server.
Examples
Token usage summary:
python scripts/fetch_metrics.py -s http://localhost:5000 -x 1 -m total_tokens -a SUM,AVG
Output: AVG: 223.91 SUM: 7613
Hourly token trend (last 24h):
python scripts/fetch_metrics.py -s http://localhost:5000 -x 1 -m total_tokens -a SUM \
-t 3600 --start-time="-24h" --end-time=now
Output: Time-bucketed token sums per hour
Latency percentiles by trace:
python scripts/fetch_metrics.py -s http://localhost:5000 -x 1 -m latency -a AVG,P95 -d trace_name
Error rate by status:
python scripts/fetch_metrics.py -s http://localhost:5000 -x 1 -m trace_count -a COUNT -d trace_status
Quality scores by evaluator (assessments):
python scripts/fetch_metrics.py -s http://localhost:5000 -x 1 -v ASSESSMENTS \
-m assessment_value -a AVG,P50 -d assessment_name
Output: Average and median scores for each evaluator (e.g., correctness, relevance)
Assessment count by name:
python scripts/fetch_metrics.py -s http://localhost:5000 -x 1 -v ASSESSMENTS \
-m assessment_count -a COUNT -d assessment_name
JSON output: Add -o json to any command.
Arguments
| Arg | Required | Description |
|---|---|---|
-s, --server | Yes | MLflow server URL |
-x, --experiment-ids | Yes | Experiment IDs (comma-separated) |
-m, --metric | Yes | trace_count, latency, input_tokens, output_tokens, total_tokens |
-a, --aggregations | Yes | COUNT, SUM, AVG, MIN, MAX, P50, P95, P99 |
-d, --dimensions | No | Group by: trace_name, trace_status |
-t, --time-interval | No | Bucket size in seconds (3600=hourly, 86400=daily) |
--start-time | No | -24h, -7d, now, ISO 8601, or epoch ms |
--end-time | No | Same formats as start-time |
-o, --output | No | table (default) or json |
For SPANS metrics (span_count, latency), add -v SPANS.
For ASSESSMENTS metrics, add -v ASSESSMENTS.
See references/api_reference.md for filter syntax and full API details.
When not to use it
- →When the task does not involve querying MLflow tracking servers
- →When the metrics needed are not related to traces, token usage, latency, or quality evaluations
- →When the MLflow server URL is not available
Limitations
- →The skill requires an MLflow tracking server URL
- →The skill is limited to the metrics and aggregations supported by `fetch_metrics.py`
- →The skill's output format is either table or JSON
How it compares
This skill provides a standardized script and command-line interface for querying specific MLflow metrics, offering structured access to data compared to manual database queries or custom API calls.
Compared to similar skills
querying-mlflow-metrics side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| querying-mlflow-metrics (this skill) | 0 | 5mo | Review | Beginner |
| weights-and-biases | 3 | 7mo | Review | Intermediate |
| rebuild-leaderboard | 0 | 1mo | Review | Advanced |
| quant-analyst | 103 | 2mo | No flags | Advanced |
Try saying
Example prompts that trigger this skill in your AI assistant.
You might also like
weights-and-biases
davila7
Track ML experiments with automatic logging, visualize training in real-time, optimize hyperparameters with sweeps, and manage model registry with W&B - collaborative MLOps platform
rebuild-leaderboard
surus-lat
Re-run all models on the LatamBoard leaderboard from scratch after data loss. Identifies which model configs exist, runs each one with the full latam_board task suite on the cluster, and publishes results to HuggingFace after each model so progress is never lost.
quant-analyst
zenobi-us
Expert quantitative analyst specializing in financial modeling, algorithmic trading, and risk analytics. Masters statistical methods, derivatives pricing, and high-frequency trading with focus on mathematical rigor, performance optimization, and profitable strategy development.
umap-learn
K-Dense-AI
UMAP dimensionality reduction. Fast nonlinear manifold learning for 2D/3D visualization, clustering preprocessing (HDBSCAN), supervised/parametric UMAP, for high-dimensional data.
embedding-strategies
wshobson
Select and optimize embedding models for semantic search and RAG applications. Use when choosing embedding models, implementing chunking strategies, or optimizing embedding quality for specific domains.
building-automl-pipelines
jeremylongshore
Build automated machine learning pipelines, including feature engineering, model selection, and performance evaluation.