langfuse-ci-integration
Connects Langfuse to GitHub Actions for automated LLM quality gates and prompt regression testing.
Install
mkdir -p .claude/skills/langfuse-ci-integration && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/2749" && unzip -o skill.zip -d .claude/skills/langfuse-ci-integration && rm skill.zipInstalls to .claude/skills/langfuse-ci-integration
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Configure Langfuse CI/CD integration with GitHub Actions and automatedKey capabilities
- →Configure GitHub Actions workflow for AI quality tests
- →Implement prompt regression tests
- →Set up experiment-driven quality gates
- →Automate prompt deployment from version control
- →Monitor score regression for AI models
How it works
The skill integrates Langfuse into CI/CD pipelines using GitHub Actions to run automated tests that validate AI model outputs and prompt quality. It uses Langfuse tracing to monitor quality gates and deploy prompts from version control.
Inputs & outputs
When to use langfuse-ci-integration
- →Set up prompt regression tests
- →Automate AI quality gates
- →Validate traces in CI/CD
- →Integrate Langfuse with GitHub Actions
About langfuse-ci-integration
Configures CI workflows to run automated tests that validate AI model outputs. It uses Langfuse tracing to monitor quality gates during pull requests.
Configure Langfuse CI/CD integration with GitHub Actions and automated testing. Use when setting up automated testing, configuring CI pipelines, or integrating Langfuse tests into your build process. Trigger with phrases like "langfuse CI", "langfuse GitHub Actions", "langfuse automated tests", "CI langfuse", "langfuse pipeline".
Prerequisites
Limitations
- →Traces may not appear in dashboard without explicit flush
- →Quality tests can be flaky with non-deterministic LLMs
- →Prompts may not be found if not deployed before tests
How it compares
This skill automates AI model and prompt validation within a CI/CD pipeline using Langfuse, providing continuous quality assurance unlike manual testing or ad-hoc evaluations.
Compared to similar skills
langfuse-ci-integration side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| langfuse-ci-integration (this skill) | 2 | 4mo | Review | Advanced |
| openevidence-ci-integration | 0 | 4mo | Review | Intermediate |
| e2e-testing-patterns | 8 | 5mo | No flags | Intermediate |
| swapper-integration | 6 | 8mo | Caution | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
More by jeremylongshore
View all by jeremylongshore →You might also like
openevidence-ci-integration
jeremylongshore
Integrate OpenEvidence testing into CI/CD pipelines. Use when setting up automated testing, configuring GitHub Actions, or implementing continuous integration for clinical AI applications. Trigger with phrases like "openevidence ci", "openevidence github actions", "openevidence pipeline", "test openevidence ci", "automate openevidence tests".
e2e-testing-patterns
wshobson
Master end-to-end testing with Playwright and Cypress to build reliable test suites that catch bugs, improve confidence, and enable fast deployment. Use when implementing E2E tests, debugging flaky tests, or establishing testing standards.
swapper-integration
shapeshift
Integrate new DEX aggregators, swappers, or bridge protocols (like Bebop, Portals, Jupiter, 0x, 1inch, etc.) into ShapeShift Web. Activates when user wants to add, integrate, or implement support for a new swapper. Guides through research, implementation, and testing following established patterns.
testing-workflow
amo-tech-ai
Comprehensive testing workflow for E2E, integration, and unit tests. Use when testing applications layer-by-layer, validating user journeys, or running test suites.
smithery-mcp-deployment
CaullenOmdahl
Best practices for creating, optimizing, and deploying MCP servers to Smithery. Use this skill when:(1) Creating new MCP servers for Smithery deployment(2) Optimizing quality scores (achieving 90/100)(3) Troubleshooting deployment issues (0/0 tools, missing annotations, low scores)(4) Migrating existing MCP servers to Smithery format(5) Understanding Smithery's schema format requirements(6) Adding workflow prompts, tool annotations, or documentation resources(7) Configuring smithery.yaml and package.json for deployment
perf-lighthouse
tech-leads-club
Run Lighthouse audits locally via CLI or Node API, parse and interpret reports, set performance budgets. Use when measuring site performance, understanding Lighthouse scores, setting up budgets, or integrating audits into CI. Triggers on: lighthouse, run lighthouse, lighthouse score, performance audit, performance budget.