Provides insight into Apache Beam's CI/CD infrastructure and GitHub Actions test workflows.
Install
mkdir -p .claude/skills/ci-cd-apache && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/17524" && unzip -o skill.zip -d .claude/skills/ci-cd-apache && rm skill.zipInstalls to .claude/skills/ci-cd-apache
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Guides understanding and working with Apache Beam's CI/CD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration.Key capabilities
- →Understand Apache Beam's CI/CD system using GitHub Actions
- →Debug CI failures by locating relevant workflow files
- →Identify PreCommit and PostCommit workflow types
- →Understand test workflows for Java, Python, and Go
- →Trigger specific workflows manually
- →Analyze common failure patterns like flaky tests or timeouts
How it works
This skill guides users through Apache Beam's GitHub Actions-based CI/CD system. It explains workflow types, key workflows, and how to trigger tests and understand test results.
Inputs & outputs
When to use ci-cd
- →Debug CI build failure
- →Understand test workflow
- →Modify CI configuration
- →Locate PreCommit tests
About this skill
CI/CD in Apache Beam
Overview
Apache Beam uses GitHub Actions for CI/CD. Workflows are located in .github/workflows/.
Workflow Types
PreCommit Workflows
- Run on PRs and merges
- Validate code changes before merge
- Naming:
beam_PreCommit_*.yml
PostCommit Workflows
- Run after merge and on schedule
- More comprehensive testing
- Naming:
beam_PostCommit_*.yml
Scheduled Workflows
- Run nightly on master
- Check for external dependency impacts
- Tag master with
nightly-master
Key Workflows
PreCommit
| Workflow | Description |
|---|---|
beam_PreCommit_Java.yml | Java build and tests |
beam_PreCommit_Python.yml | Python tests |
beam_PreCommit_Go.yml | Go tests |
beam_PreCommit_RAT.yml | License header checks |
beam_PreCommit_Spotless.yml | Code formatting |
PostCommit - Java
| Workflow | Description |
|---|---|
beam_PostCommit_Java.yml | Full Java test suite |
beam_PostCommit_Java_ValidatesRunner_*.yml | Runner validation tests |
beam_PostCommit_Java_Examples_*.yml | Example pipeline tests |
PostCommit - Python
| Workflow | Description |
|---|---|
beam_PostCommit_Python.yml | Full Python test suite |
beam_PostCommit_Python_ValidatesRunner_*.yml | Runner validation |
beam_PostCommit_Python_Examples_*.yml | Examples |
Load & Performance Tests
| Workflow | Description |
|---|---|
beam_LoadTests_*.yml | Load testing |
beam_PerformanceTests_*.yml | I/O performance |
Triggering Tests
Automatic
- PRs trigger PreCommit tests
- Merges trigger PostCommit tests
Triggering Specific Workflows
Use trigger files to run specific workflows.
Workflow Dispatch
Most workflows support manual triggering via GitHub UI.
Understanding Test Results
Finding Logs
- Go to PR → Checks tab
- Click on failed workflow
- Expand failed job
- View step logs
Common Failure Patterns
Flaky Tests
- Random failures unrelated to change
- Solution: Use trigger files to re-run the specific workflow.
Timeout
- Increase timeout in workflow if justified
- Or optimize test
Resource Exhaustion
- GCP quota issues
- Check project settings
GCP Credentials
Workflows requiring GCP access use these secrets:
GCP_PROJECT_ID- Project ID (e.g.,apache-beam-testing)GCP_REGION- Region (e.g.,us-central1)GCP_TESTING_BUCKET- Temp storage bucketGCP_PYTHON_WHEELS_BUCKET- Python wheels bucketGCP_SA_EMAIL- Service account emailGCP_SA_KEY- Base64-encoded service account key
Required IAM roles:
- Storage Admin
- Dataflow Admin
- Artifact Registry Writer
- BigQuery Data Editor
- Service Account User
Self-hosted vs GitHub-hosted Runners
Self-hosted (majority of workflows)
- Pre-configured with dependencies
- GCP credentials pre-configured
- Naming:
beam_*.yml
GitHub-hosted
- Used for cross-platform testing (Linux, macOS, Windows)
- May need explicit credential setup
Workflow Structure
name: Workflow Name
on:
push:
branches: [master]
pull_request:
branches: [master]
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
jobs:
build:
runs-on: [self-hosted, ...]
steps:
- uses: actions/checkout@v4
- name: Run Gradle
run: ./gradlew :task:name
Local Debugging
Run Same Commands as CI
Check workflow file's run commands:
./gradlew :sdks:java:core:test
./gradlew :sdks:python:test
Common Issues
- Clean gradle cache:
rm -rf ~/.gradle .gradle - Remove build directory:
rm -rf build - Check Java version matches CI
Snapshot Builds
Locations
- Java SDK: https://repository.apache.org/content/groups/snapshots/org/apache/beam/
- SDK Containers: https://gcr.io/apache-beam-testing/beam-sdk
- Portable Runners: https://gcr.io/apache-beam-testing/beam_portability
- Python SDK: gs://beam-python-nightly-snapshots
Release Workflows
| Workflow | Purpose |
|---|---|
cut_release_branch.yml | Create release branch |
build_release_candidate.yml | Build RC |
finalize_release.yml | Finalize release |
publish_github_release_notes.yml | Publish notes |
When not to use it
- →When the project does not use GitHub Actions for CI/CD
Limitations
- →Specific to Apache Beam's CI/CD system
- →Focuses on GitHub Actions workflows
How it compares
This skill provides a structured guide to Apache Beam's specific CI/CD implementation, offering targeted debugging and configuration insights beyond general GitHub Actions documentation.
Compared to similar skills
ci-cd side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| ci-cd (this skill) | 0 | 5mo | Review | Intermediate |
| e2e-testing-patterns | 8 | 2mo | No flags | Intermediate |
| testing-workflow | 16 | 9mo | Review | Intermediate |
| perf-lighthouse | 13 | 5mo | Review | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
More by apache
View all by apache →You might also like
e2e-testing-patterns
wshobson
Master end-to-end testing with Playwright and Cypress to build reliable test suites that catch bugs, improve confidence, and enable fast deployment. Use when implementing E2E tests, debugging flaky tests, or establishing testing standards.
testing-workflow
amo-tech-ai
Comprehensive testing workflow for E2E, integration, and unit tests. Use when testing applications layer-by-layer, validating user journeys, or running test suites.
perf-lighthouse
tech-leads-club
Run Lighthouse audits locally via CLI or Node API, parse and interpret reports, set performance budgets. Use when measuring site performance, understanding Lighthouse scores, setting up budgets, or integrating audits into CI. Triggers on: lighthouse, run lighthouse, lighthouse score, performance audit, performance budget.
xcodebuildmcp
cameroncooke
Official skill for XcodeBuildMCP. Use when doing iOS/macOS/watchOS/tvOS/visionOS work (build, test, run, debug, log, UI automation).
verify
Use when you want to validate changes before committing, or when you need to check all React contribution requirements.
playwright-pro
alirezarezvani
Production-grade Playwright testing toolkit. Use when the user mentions Playwright tests, end-to-end testing, browser automation, fixing flaky tests, test migration, CI/CD testing, or test suites. Generate tests, fix flaky failures, migrate from Cypress/Selenium, sync with TestRail, run on BrowserStack. 55 templates, 3 agents, smart reporting.