LA

langfuse-ci-integration

Connects Langfuse to GitHub Actions for automated LLM quality gates and prompt regression testing.

Install

mkdir -p .claude/skills/langfuse-ci-integration && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/2749" && unzip -o skill.zip -d .claude/skills/langfuse-ci-integration && rm skill.zip

Installs to .claude/skills/langfuse-ci-integration

Activation

This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.

Configure Langfuse CI/CD integration with GitHub Actions and automated
70 charsno explicit “when” trigger
Advanced

Key capabilities

  • →Configure GitHub Actions workflow for AI quality tests
  • →Implement prompt regression tests
  • →Set up experiment-driven quality gates
  • →Automate prompt deployment from version control
  • →Monitor score regression for AI models

How it works

The skill integrates Langfuse into CI/CD pipelines using GitHub Actions to run automated tests that validate AI model outputs and prompt quality. It uses Langfuse tracing to monitor quality gates and deploy prompts from version control.

Inputs & outputs

You give it
AI model code, prompt definitions, test datasets, GitHub pull requests
You get back
AI quality test results, prompt regression test outcomes, experiment accuracy scores, deployed prompts, average quality scores

When to use langfuse-ci-integration

  • →Set up prompt regression tests
  • →Automate AI quality gates
  • →Validate traces in CI/CD
  • →Integrate Langfuse with GitHub Actions

About langfuse-ci-integration

Configures CI workflows to run automated tests that validate AI model outputs. It uses Langfuse tracing to monitor quality gates during pull requests.

Configure Langfuse CI/CD integration with GitHub Actions and automated testing. Use when setting up automated testing, configuring CI pipelines, or integrating Langfuse tests into your build process. Trigger with phrases like "langfuse CI", "langfuse GitHub Actions", "langfuse automated tests", "CI langfuse", "langfuse pipeline".

Prerequisites

Langfuse API keys stored as GitHub secretsTest framework (Vitest or Jest)OpenAI API key for LLM tests

Limitations

  • →Traces may not appear in dashboard without explicit flush
  • →Quality tests can be flaky with non-deterministic LLMs
  • →Prompts may not be found if not deployed before tests

How it compares

This skill automates AI model and prompt validation within a CI/CD pipeline using Langfuse, providing continuous quality assurance unlike manual testing or ad-hoc evaluations.

Compared to similar skills

langfuse-ci-integration side by side with the closest alternatives in the catalog.

SkillInstallsUpdatedSafetyDifficulty
langfuse-ci-integration (this skill)24moReviewAdvanced
openevidence-ci-integration04moReviewIntermediate
e2e-testing-patterns85moNo flagsIntermediate
swapper-integration68moCautionIntermediate

Try saying

Example prompts that trigger this skill in your AI assistant.

More by jeremylongshore

View all by jeremylongshore →

analyzing-logs

jeremylongshore

Analyze application logs to detect performance issues, identify error patterns, and improve stability by extracting key insights.

14123

ollama-setup

jeremylongshore

Configure auto-configure Ollama when user needs local LLM deployment, free AI alternatives, or wants to eliminate hosted API costs. Trigger phrases: "install ollama", "local AI", "free LLM", "self-hosted AI", "replace OpenAI", "no API costs". Use when appropriate context detected. Trigger with relevant phrases based on skill purpose.

1167

backtesting-trading-strategies

jeremylongshore

Backtest crypto and traditional trading strategies against historical data. Calculates performance metrics (Sharpe, Sortino, max drawdown), generates equity curves, and optimizes strategy parameters. Use when user wants to test a trading strategy, validate signals, or compare approaches. Trigger with phrases like "backtest strategy", "test trading strategy", "historical performance", "simulate trades", "optimize parameters", or "validate signals".

1071

generating-database-seed-data

jeremylongshore

Process this skill enables AI assistant to generate realistic test data and database seed scripts for development and testing environments. it uses faker libraries to create realistic data, maintains relational integrity, and allows configurable data volumes. u... Use when working with databases or data models. Trigger with phrases like 'database', 'query', or 'schema'.

1033

cursor-codebase-indexing

jeremylongshore

Execute set up and optimize Cursor codebase indexing. Triggers on "cursor index setup", "codebase indexing", "index codebase", "cursor semantic search". Use when working with cursor codebase indexing functionality. Trigger with phrases like "cursor codebase indexing", "cursor indexing", "cursor".

885

testing-mobile-apps

jeremylongshore

Execute mobile app testing on iOS and Android devices/simulators. Use when performing specialized testing. Trigger with phrases like "test mobile app", "run iOS tests", or "validate Android functionality".

810

You might also like

openevidence-ci-integration

jeremylongshore

Integrate OpenEvidence testing into CI/CD pipelines. Use when setting up automated testing, configuring GitHub Actions, or implementing continuous integration for clinical AI applications. Trigger with phrases like "openevidence ci", "openevidence github actions", "openevidence pipeline", "test openevidence ci", "automate openevidence tests".

00

e2e-testing-patterns

wshobson

Master end-to-end testing with Playwright and Cypress to build reliable test suites that catch bugs, improve confidence, and enable fast deployment. Use when implementing E2E tests, debugging flaky tests, or establishing testing standards.

8102

swapper-integration

shapeshift

Integrate new DEX aggregators, swappers, or bridge protocols (like Bebop, Portals, Jupiter, 0x, 1inch, etc.) into ShapeShift Web. Activates when user wants to add, integrate, or implement support for a new swapper. Guides through research, implementation, and testing following established patterns.

696

testing-workflow

amo-tech-ai

Comprehensive testing workflow for E2E, integration, and unit tests. Use when testing applications layer-by-layer, validating user journeys, or running test suites.

1683

smithery-mcp-deployment

CaullenOmdahl

Best practices for creating, optimizing, and deploying MCP servers to Smithery. Use this skill when:(1) Creating new MCP servers for Smithery deployment(2) Optimizing quality scores (achieving 90/100)(3) Troubleshooting deployment issues (0/0 tools, missing annotations, low scores)(4) Migrating existing MCP servers to Smithery format(5) Understanding Smithery's schema format requirements(6) Adding workflow prompts, tool annotations, or documentation resources(7) Configuring smithery.yaml and package.json for deployment

881

perf-lighthouse

tech-leads-club

Run Lighthouse audits locally via CLI or Node API, parse and interpret reports, set performance budgets. Use when measuring site performance, understanding Lighthouse scores, setting up budgets, or integrating audits into CI. Triggers on: lighthouse, run lighthouse, lighthouse score, performance audit, performance budget.

1361

Search skills

Search the agent skills registry