senior-data-scientist
A data science assistant for statistical modeling, experimentation, and production ML pipelines using Python, SQL, and R.
Install
mkdir -p .claude/skills/senior-data-scientist && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/772" && unzip -o skill.zip -d .claude/skills/senior-data-scientist && rm skill.zipInstalls to .claude/skills/senior-data-scientist
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
World-class data science skill for statistical modeling, experimentation, causal inference, and advanced analytics. Expertise in Python (NumPy, Pandas, Scikit-learn), R, SQL, statistical methods, A/B testing, time series, and business intelligence. Includes experiment design, feature engineering, model evaluation, and stakeholder communication. Use when designing experiments, building predictive models, performing causal analysis, or driving data-driven decisions.Key capabilities
- →Design production-grade experiment frameworks
- →Execute feature engineering pipelines
- →Perform model evaluation and validation
- →Implement real-time inference workflows
- →Manage distributed computing architectures
How it works
The skill utilizes specialized Python scripts to execute experiment design, feature engineering, and model evaluation suites. It integrates with MLOps tools to manage the lifecycle from training to production deployment.
Inputs & outputs
When to use senior-data-scientist
- →Design and analyze A/B tests
- →Build predictive machine learning models
- →Perform causal inference and statistical analysis
- →Optimize production ML pipelines
About this skill
Senior Data Scientist
World-class senior data scientist skill for production-grade AI/ML/Data systems.
Quick Start
Main Capabilities
# Core Tool 1
python scripts/experiment_designer.py --input data/ --output results/
# Core Tool 2
python scripts/feature_engineering_pipeline.py --target project/ --analyze
# Core Tool 3
python scripts/model_evaluation_suite.py --config config.yaml --deploy
Core Expertise
This skill covers world-class capabilities in:
- Advanced production patterns and architectures
- Scalable system design and implementation
- Performance optimization at scale
- MLOps and DataOps best practices
- Real-time processing and inference
- Distributed computing frameworks
- Model deployment and monitoring
- Security and compliance
- Cost optimization
- Team leadership and mentoring
Tech Stack
Languages: Python, SQL, R, Scala, Go ML Frameworks: PyTorch, TensorFlow, Scikit-learn, XGBoost Data Tools: Spark, Airflow, dbt, Kafka, Databricks LLM Frameworks: LangChain, LlamaIndex, DSPy Deployment: Docker, Kubernetes, AWS/GCP/Azure Monitoring: MLflow, Weights & Biases, Prometheus Databases: PostgreSQL, BigQuery, Snowflake, Pinecone
Reference Documentation
1. Statistical Methods Advanced
Comprehensive guide available in references/statistical_methods_advanced.md covering:
- Advanced patterns and best practices
- Production implementation strategies
- Performance optimization techniques
- Scalability considerations
- Security and compliance
- Real-world case studies
2. Experiment Design Frameworks
Complete workflow documentation in references/experiment_design_frameworks.md including:
- Step-by-step processes
- Architecture design patterns
- Tool integration guides
- Performance tuning strategies
- Troubleshooting procedures
3. Feature Engineering Patterns
Technical reference guide in references/feature_engineering_patterns.md with:
- System design principles
- Implementation examples
- Configuration best practices
- Deployment strategies
- Monitoring and observability
Production Patterns
Pattern 1: Scalable Data Processing
Enterprise-scale data processing with distributed computing:
- Horizontal scaling architecture
- Fault-tolerant design
- Real-time and batch processing
- Data quality validation
- Performance monitoring
Pattern 2: ML Model Deployment
Production ML system with high availability:
- Model serving with low latency
- A/B testing infrastructure
- Feature store integration
- Model monitoring and drift detection
- Automated retraining pipelines
Pattern 3: Real-Time Inference
High-throughput inference system:
- Batching and caching strategies
- Load balancing
- Auto-scaling
- Latency optimization
- Cost optimization
Best Practices
Development
- Test-driven development
- Code reviews and pair programming
- Documentation as code
- Version control everything
- Continuous integration
Production
- Monitor everything critical
- Automate deployments
- Feature flags for releases
- Canary deployments
- Comprehensive logging
Team Leadership
- Mentor junior engineers
- Drive technical decisions
- Establish coding standards
- Foster learning culture
- Cross-functional collaboration
Performance Targets
Latency:
- P50: < 50ms
- P95: < 100ms
- P99: < 200ms
Throughput:
- Requests/second: > 1000
- Concurrent users: > 10,000
Availability:
- Uptime: 99.9%
- Error rate: < 0.1%
Security & Compliance
- Authentication & authorization
- Data encryption (at rest & in transit)
- PII handling and anonymization
- GDPR/CCPA compliance
- Regular security audits
- Vulnerability management
Common Commands
# Development
python -m pytest tests/ -v --cov
python -m black src/
python -m pylint src/
# Training
python scripts/train.py --config prod.yaml
python scripts/evaluate.py --model best.pth
# Deployment
docker build -t service:v1 .
kubectl apply -f k8s/
helm upgrade service ./charts/
# Monitoring
kubectl logs -f deployment/service
python scripts/health_check.py
Resources
- Advanced Patterns:
references/statistical_methods_advanced.md - Implementation Guide:
references/experiment_design_frameworks.md - Technical Reference:
references/feature_engineering_patterns.md - Automation Scripts:
scripts/directory
Senior-Level Responsibilities
As a world-class senior professional:
-
Technical Leadership
- Drive architectural decisions
- Mentor team members
- Establish best practices
- Ensure code quality
-
Strategic Thinking
- Align with business goals
- Evaluate trade-offs
- Plan for scale
- Manage technical debt
-
Collaboration
- Work across teams
- Communicate effectively
- Build consensus
- Share knowledge
-
Innovation
- Stay current with research
- Experiment with new approaches
- Contribute to community
- Drive continuous improvement
-
Production Excellence
- Ensure high availability
- Monitor proactively
- Optimize performance
- Respond to incidents
When not to use it
- →Small-scale projects without production requirements
- →Tasks requiring only basic descriptive statistics
Prerequisites
Limitations
- →Requires specific directory structures for scripts
- →Dependent on external ML frameworks like PyTorch or TensorFlow
How it compares
Unlike manual data analysis, this skill provides standardized, production-ready pipelines for scalable system architecture and automated model monitoring.
Compared to similar skills
senior-data-scientist side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| senior-data-scientist (this skill) | 9 | 7mo | Review | Advanced |
| hugging-face-datasets | 1 | 6mo | Review | Intermediate |
| elo-engineering | 0 | 3mo | No flags | Intermediate |
| rag-index | 0 | 6mo | Review | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
More by davila7
View all by davila7 →You might also like
hugging-face-datasets
patchy631
Create and manage datasets on Hugging Face Hub. Supports initializing repos, defining configs/system prompts, streaming row updates, and SQL-based dataset querying/transformation. Designed to work alongside HF MCP server for comprehensive dataset workflows.
elo-engineering
sports-data-hq
Builds multi-variant Elo rating systems for sports prediction. Use when user asks about Elo ratings, rating systems, team strength, K-factor, home field advantage, season carryover, margin-of-victory adjustments, or mentions 'Fading Elo', 'Form Elo', 'Component Elo'. Do not use for general feature c
rag-index
brifl
Experimental RAG indexing utilities (scanner + indexer + retriever).
feast-user-guide
feast-dev
Guide for working with Feast (Feature Store) — defining features, configuring feature_store.yaml, retrieving features online/offline, using the CLI, and building RAG retrieval pipelines. Use when the user asks about creating entities, feature views, on-demand feature views, stream feature views, fea
cocoindex
cocoindex-io
Comprehensive toolkit for developing with the CocoIndex library. Use when users need to create data transformation pipelines (flows), write custom functions, or operate flows via CLI or API. Covers building ETL workflows for AI data processing, including embedding documents into vector databases, building knowledge graphs, creating search indexes, or processing data streams with incremental updates.
similarity-search-patterns
wshobson
Implement efficient similarity search with vector databases. Use when building semantic search, implementing nearest neighbor queries, or optimizing retrieval performance.