vector-index-tuning
Tune and scale vector index performance.
Install
mkdir -p .claude/skills/vector-index-tuning && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/730" && unzip -o skill.zip -d .claude/skills/vector-index-tuning && rm skill.zipInstalls to .claude/skills/vector-index-tuning
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Optimize vector index performance for latency, recall, and memory. Use when tuning HNSW parameters, selecting quantization strategies, or scaling vector search infrastructure.Key capabilities
- →Tunes HNSW parameters
- →Selects vector quantization methods
- →Optimizes index memory footprint
- →Balances search latency versus recall
How it works
Applies established architectural thresholds to map dataset size and latency targets to specific HNSW and quantization configurations.
Inputs & outputs
When to use vector-index-tuning
- →Tune HNSW parameters for better recall
- →Implement quantization to reduce memory footprint
- →Optimize vector search latency for large datasets
About this skill
Vector Index Tuning
Guide to optimizing vector indexes for production performance.
When to Use This Skill
- Tuning HNSW parameters
- Implementing quantization
- Optimizing memory usage
- Reducing search latency
- Balancing recall vs speed
- Scaling to billions of vectors
Core Concepts
1. Index Type Selection
Data Size Recommended Index
────────────────────────────────────────
< 10K vectors → Flat (exact search)
10K - 1M → HNSW
1M - 100M → HNSW + Quantization
> 100M → IVF + PQ or DiskANN
2. HNSW Parameters
| Parameter | Default | Effect |
|---|---|---|
| M | 16 | Connections per node, ↑ = better recall, more memory |
| efConstruction | 100 | Build quality, ↑ = better index, slower build |
| efSearch | 50 | Search quality, ↑ = better recall, slower search |
3. Quantization Types
Full Precision (FP32): 4 bytes × dimensions
Half Precision (FP16): 2 bytes × dimensions
INT8 Scalar: 1 byte × dimensions
Product Quantization: ~32-64 bytes total
Binary: dimensions/8 bytes
Templates and detailed worked examples
Full template library and detailed worked examples live in references/details.md. Read that file when you need the concrete templates.
Best Practices
Do's
- Benchmark with real queries - Synthetic may not represent production
- Monitor recall continuously - Can degrade with data drift
- Start with defaults - Tune only when needed
- Use quantization - Significant memory savings
- Consider tiered storage - Hot/cold data separation
Don'ts
- Don't over-optimize early - Profile first
- Don't ignore build time - Index updates have cost
- Don't forget reindexing - Plan for maintenance
- Don't skip warming - Cold indexes are slow
When not to use it
- →Small-scale datasets (<10k vectors)
- →Real-time non-vector search requirements
Limitations
- →Requires empirical performance benchmarks
- →Quantization settings depend on specific vector storage
How it compares
It offers quantitative mappings for index parameters instead of trial-and-error optimization.
Compared to similar skills
vector-index-tuning side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| vector-index-tuning (this skill) | 5 | 2mo | No flags | Advanced |
| quant-analyst | 103 | 2mo | No flags | Advanced |
| umap-learn | 6 | 2mo | Review | Intermediate |
| embedding-strategies | 8 | 2mo | No flags | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
More by wshobson
View all by wshobson →You might also like
quant-analyst
zenobi-us
Expert quantitative analyst specializing in financial modeling, algorithmic trading, and risk analytics. Masters statistical methods, derivatives pricing, and high-frequency trading with focus on mathematical rigor, performance optimization, and profitable strategy development.
umap-learn
K-Dense-AI
UMAP dimensionality reduction. Fast nonlinear manifold learning for 2D/3D visualization, clustering preprocessing (HDBSCAN), supervised/parametric UMAP, for high-dimensional data.
embedding-strategies
wshobson
Select and optimize embedding models for semantic search and RAG applications. Use when choosing embedding models, implementing chunking strategies, or optimizing embedding quality for specific domains.
building-automl-pipelines
jeremylongshore
Build automated machine learning pipelines, including feature engineering, model selection, and performance evaluation.
model-compare
rawwerks
Compare 3D CAD models using boolean operations (IoU, Dice, precision/recall). Use when evaluating generated models against gold references, diffing CAD revisions, or computing similarity metrics for ML training. Triggers on: model diff, compare models, IoU, intersection over union, model similarity, CAD comparison, STEP diff, 3D evaluation, gold reference, generated model, precision recall 3D.
matchms
davila7
Mass spectrometry analysis. Process mzML/MGF/MSP, spectral similarity (cosine, modified cosine), metadata harmonization, compound ID, for metabolomics and MS data processing.