data-engineering
Automatically reviews SQL performance and data pipeline designs for optimization.
Install
mkdir -p .claude/skills/data-engineering-mazelb && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/18186" && unzip -o skill.zip -d .claude/skills/data-engineering-mazelb && rm skill.zipInstalls to .claude/skills/data-engineering-mazelb
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Automatically reviews data pipelines, optimizes SQL queries, and ensures data quality. Activates when you mention: data pipeline, ETL, SQL optimization, query performance, data quality, schema design, Airflow DAG, database index, slow query. Provides: Pipeline design patterns, SQL query optimization, index recommendations, data validation strategies, schema improvements. Supports: PostgreSQL, MySQL, SQLAlchemy, Prisma, Spring Data JPAKey capabilities
- →Identify N+1 query problems, missing indexes, and inefficient JOINs in SQL queries.
- →Suggest best practices for idempotency, error handling, and monitoring in data pipeline design.
- →Check for schema validation, null handling, and duplicate detection to ensure data quality.
- →Optimize OpenSearch indexing and batch document processing for rag-project archetypes.
- →Recommend SQLAlchemy eager loading and connection pooling for api-service archetypes.
How it works
The skill activates upon detecting specific keywords or code related to data engineering, then analyzes the input to identify performance issues, design flaws, or quality concerns.
Inputs & outputs
When to use data-engineering
- →Optimizing slow SQL queries
- →Reviewing data pipeline performance
- →Improving database schema design
About this skill
Data Engineering Skill
Automatically optimizes data pipelines, SQL queries, and data quality.
When This Skill Activates
Claude invokes this skill when you:
- Show SQL queries or database code
- Mention slow queries or performance
- Discuss data pipelines or ETL
- Ask about data quality
- Work with database schema
What This Skill Does
1. SQL Query Optimization
Identifies:
- N+1 query problems
- Missing indexes
- Inefficient JOINs
- Unnecessary columns in SELECT
- Lack of query limits
Example Optimization:
-- Before: Sequential scan (45ms)
SELECT * FROM users WHERE email = '[email protected]';
-- Add index
CREATE INDEX idx_users_email ON users(email);
-- After: Index scan (2ms) - 95% faster
2. Data Pipeline Design
Best Practices:
- Idempotency (safe to rerun)
- Error handling and retries
- Monitoring and alerting
- Data validation
- Incremental processing
3. Data Quality
Checks for:
- Schema validation
- Null handling
- Duplicate detection
- Referential integrity
- Data type consistency
Archetype-Specific Optimizations
For rag-project:
- OpenSearch indexing optimization
- Batch document processing
- Embedding caching strategies
- Vector search performance
For api-service:
- Database query optimization
- SQLAlchemy eager loading
- Connection pooling
- Query result caching
Output Format
Provides:
- Query Analysis: Performance issues
- Execution Plan: EXPLAIN output
- Optimized Query: Improved version
- Index Recommendations: SQL to create indexes
- Performance Gains: Expected improvements
Example Usage
User: "This query is slow: SELECT * FROM users JOIN profiles..."
Claude: [Activates data-engineering skill]
- Identifies N+1 problem
- Suggests eager loading
- Recommends indexes
- Provides optimized query
Result: 2000ms → 150ms (13x faster)
When not to use it
- →When the task does not involve SQL queries or database code.
- →When the discussion is not about data pipelines or ETL.
- →When the focus is not on data quality or schema design.
Limitations
- →The skill's support is limited to PostgreSQL, MySQL, SQLAlchemy, Prisma, and Spring Data JPA.
- →The skill focuses on specific archetypes like rag-project and api-service for specialized optimizations.
How it compares
This skill automatically provides specific optimizations and recommendations based on identified patterns, unlike a manual review that requires human expertise to diagnose and suggest improvements.
Compared to similar skills
data-engineering side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| data-engineering (this skill) | 0 | 7mo | Review | Intermediate |
| find-hypertable-candidates | 1 | 3mo | No flags | Intermediate |
| sql-queries | 18 | 4mo | No flags | Intermediate |
| fuzzy-matching | 8 | 5mo | No flags | Advanced |
Try saying
Example prompts that trigger this skill in your AI assistant.
You might also like
find-hypertable-candidates
timescale
Use this skill to analyze an existing PostgreSQL database and identify which tables should be converted to Timescale/TimescaleDB hypertables. **Trigger when user asks to:** - Analyze database tables for hypertable conversion potential - Identify time-series or event tables in an existing schema - Evaluate if a table would benefit from Timescale/TimescaleDB - Audit PostgreSQL tables for migration to Timescale/TimescaleDB/TigerData - Score or rank tables for hypertable candidacy **Keywords:** hypertable candidate, table analysis, migration assessment, Timescale, TimescaleDB, time-series detection, insert-heavy tables, event logs, audit tables Provides SQL queries to analyze table statistics, index patterns, and query patterns. Includes scoring criteria (8+ points = good candidate) and pattern recognition for IoT, events, transactions, and sequential data.
sql-queries
anthropics
Write correct, performant SQL across all major data warehouse dialects (Snowflake, BigQuery, Databricks, PostgreSQL, etc.). Use when writing queries, optimizing slow SQL, translating between dialects, or building complex analytical queries with CTEs, window functions, or aggregations.
fuzzy-matching
dadbodgeoff
Multi-stage fuzzy matching pipeline for entity reconciliation. PostgreSQL trigram pre-filter, salient overlap check, and multi-factor similarity scoring.
sql-pro
sickn33
Master modern SQL with cloud-native databases, OLTP/OLAP optimization, and advanced query techniques. Expert in performance tuning, data modeling, and hybrid analytical systems. Use PROACTIVELY for database optimization or complex analysis.
postgres-hybrid-text-search
timescale
Use this skill to implement hybrid search combining BM25 keyword search with semantic vector search using Reciprocal Rank Fusion (RRF). **Trigger when user asks to:** - Combine keyword and semantic search - Implement hybrid search or multi-modal retrieval - Use BM25/pg_textsearch with pgvector together - Implement RRF (Reciprocal Rank Fusion) for search - Build search that handles both exact terms and meaning **Keywords:** hybrid search, BM25, pg_textsearch, RRF, reciprocal rank fusion, keyword search, full-text search, reranking, cross-encoder Covers: pg_textsearch BM25 index setup, parallel query patterns, client-side RRF fusion (Python/TypeScript), weighting strategies, and optional ML reranking.
whodb
clidey
Database operations including querying, schema exploration, and data analysis. Activates for tasks involving PostgreSQL, MySQL, MariaDB, SQLite, MongoDB, Redis, Elasticsearch, or ClickHouse databases.