codeql

CodeQL Analysis

Supported languages: Python, JavaScript/TypeScript, Go, Java/Kotlin, C/C++, C#, Ruby, Swift.

Skill resources: Reference files and templates are located at {baseDir}/references/ and {baseDir}/workflows/.

Essential Principles

1. Database quality is non-negotiable. A database that builds is not automatically good — a cached build extracts nothing while reporting success.

2. Data extensions catch what CodeQL misses. Django, Spring, and Express projects still wrap database calls, request parsing, and shell execution in project-specific APIs that no shipped model covers.

3. Explicit suite references prevent silent query dropping. Never pass pack names to codeql database analyze — each pack's defaultSuiteFile applies hidden filters that can produce zero results. Always generate a .qls.

4. Zero findings needs investigation, not celebration. It can mean poor extraction, missing models, the wrong packs, or suite filtering. Run {baseDir}/scripts/check_db_quality.py after the build, confirm {baseDir}/scripts/verify_query_suite.py exited zero for the suite in use — the generation scripts run it, so invoke it by hand only for a reused or hand-edited suite — and say in the report that both passed.

5. macOS Apple Silicon requires workarounds for compiled languages. Exit code 137 is an arm64e/arm64 mismatch, not a build failure. Try Homebrew arm64 tools or Rosetta before falling back to build-mode=none.

6. Follow workflows step by step. Each phase gates the next; skipping quality assessment or data extensions leaves the gap invisible in the results.

Each Bash call is a fresh shell

Nothing carries across a Bash call: not variables, not arrays, not functions sourced from build_log.sh. Every block below that uses a value must re-establish it in the same block. The workflows point back here rather than repeating it; what they do state is the specific damage at that site, because each one fails differently and silently:

• a lost function makes run_logged exit 127, which the build ladder reads as a failed method and walks down to --build-mode=none, never having invoked CodeQL
• a lost array expands to nothing, so every --threat-model and --model-packs the user chose is dropped while the final report still lists them as used
• a lost scalar under set -u aborts the block with unbound variable

Output Directory

All generated files (database, build logs, diagnostics, extensions, results) are stored in a single output directory.

• If the user specifies an output directory in their prompt, use it as OUTPUT_DIR.
• If not specified, default to ./static_analysis_codeql_1. If that already exists, increment to _2, _3, etc.

In both cases, always create the directory with mkdir -p before writing any files.

Set USER_SPECIFIED_DIR to the literal path from the user's prompt before running this, or leave it unset to auto-increment. Nothing else assigns it.

# Resolve output directory
USER_SPECIFIED_DIR="${USER_SPECIFIED_DIR:-}"   # substitute the user's path here, if any
if [ -n "$USER_SPECIFIED_DIR" ]; then
  OUTPUT_DIR="$USER_SPECIFIED_DIR"
else
  BASE="static_analysis_codeql"
  N=1
  while [ -e "${BASE}_${N}" ]; do
    N=$((N + 1))
  done
  OUTPUT_DIR="${BASE}_${N}"
fi
mkdir -p "$OUTPUT_DIR"

The output directory is resolved once at the start before any workflow executes. All workflows receive $OUTPUT_DIR and store their artifacts there:

$OUTPUT_DIR/
├── rulesets.txt                 # Selected query packs (logged after Step 3)
├── codeql.db/                   # CodeQL database (dir containing codeql-database.yml)
├── build.log                    # Build log
├── codeql-config.yml            # Exclusion config (interpreted languages)
├── diagnostics/                 # Diagnostic queries and CSVs
├── extensions/                  # Data extension YAMLs
├── raw/                         # Unfiltered analysis output
│   ├── results.sarif
│   └── run-all.qls | important-only.qls
└── results/                     # Final results (filtered for important-only, copied for run-all)
    └── results.sarif

Database Discovery

A CodeQL database is identified by the presence of a codeql-database.yml marker file inside its directory. When searching for existing databases, always collect all matches — there may be multiple databases from previous runs or for different languages.

Discovery command. find_databases.sh prints one database path per line, filtering out the marker files a failed build leaves behind. Build the array in the same block that selects from it — each Bash call is a fresh shell, so an array built here is empty by the next call, and the run concludes there is no database:

# Command substitution, not `done < <(...)`: a process substitution discards the script's
# exit status, so "codeql is not on this shell's PATH" (exit 2) would arrive as an empty
# list and route to "build a new database" with three good ones sitting on disk.
if ! DB_LIST=$("{baseDir}/scripts/find_databases.sh" "${OUTPUT_DIR:-.}" .); then
  echo "ERROR: database discovery failed — see the message above" >&2
  exit 1
fi

FOUND_DBS=()
while IFS= read -r db; do
  [ -n "$db" ] || continue
  FOUND_DBS+=("$db")
done <<<"$DB_LIST"

echo "Found ${#FOUND_DBS[@]} existing database(s)"

# The metadata the selection prompt needs, collected here rather than in a block of its
# own: FOUND_DBS is gone by the next Bash call, and a loop over an array that no longer
# exists prints nothing and reports success.
for db in "${FOUND_DBS[@]}"; do
  CODEQL_LANG=$(codeql resolve database --format=json -- "$db" 2>/dev/null | jq -r '.languages[0]')
  CREATED=$(grep '^creationMetadata:' -A5 "$db/codeql-database.yml" 2>/dev/null | grep 'creationTime' | awk '{print $2}')
  echo "$db — language: $CODEQL_LANG, created: $CREATED"
done

Never assume a database is named codeql.db — discover it by its marker file.

When multiple databases are found: use AskUserQuestion to let the user select which database to use, or to build a new one, from the language and creation time printed above. AskUserQuestion takes at most four options, so with more databases than that, offer the three most recent plus "Build a new database" and list the rest in the prompt text. Skip AskUserQuestion if the user explicitly stated which database to use or to build a new one in their prompt.

Quick Start

For the common case ("scan this codebase for vulnerabilities"):

# Verify CodeQL is installed. Stop here if it is not — every later command fails with
# a less informative error, and the run wastes a build cycle before saying why.
if ! command -v codeql >/dev/null 2>&1; then
  echo "ERROR: codeql not found on PATH. Install it with one of:" >&2
  echo "  gh extension install github/gh-codeql   # then: gh codeql install-stub" >&2
  echo "  brew install --cask codeql" >&2
  echo "  https://github.com/github/codeql-action/releases  (codeql-bundle)" >&2
  exit 1
fi

# jq parses `codeql resolve database --format=json` in the very next step. Without it
# CODEQL_LANG comes back empty and the run continues against the wrong language.
if ! command -v jq >/dev/null 2>&1; then
  echo "ERROR: jq not found on PATH (brew install jq / apt install jq)" >&2
  exit 1
fi

# uv runs both guard scripts and both suite generators. Check it here rather than at
# suite generation, which is after the build — otherwise a machine without uv spends
# the whole build before failing.
if ! command -v uv >/dev/null 2>&1; then
  echo "ERROR: uv not found on PATH (https://docs.astral.sh/uv/getting-started/)" >&2
  exit 1
fi

codeql --version

Then resolve OUTPUT_DIR using the block in Output Directory above — it honours a user-specified directory, which a bare auto-increment does not.

Then execute the full pipeline: build database → create data extensions → run analysis using the workflows below.

Rationalizations to Reject

These shortcuts lead to missed findings. Do not accept them:

• "security-extended is enough" - It is the baseline. Always check if Trail of Bits packs and Community Packs are available for the language. They catch categories security-extended misses entirely.
• "security-and-quality is the broadest suite" - security-and-quality excludes all experimental/ query paths. For run-all mode, import both security-and-quality and security-experimental. The delta is 1–52 queries depending on the language.
• "The database built, so it's good" - A database that builds does not mean it extracted well. Always run quality assessment and check file counts against expected source files.
• "Data extensions aren't needed for standard frameworks" - Even Django/Spring apps have custom wrappers that CodeQL does not model. Skipping extensions means missing vulnerabilities.
• "build-mode=none is fine for compiled languages" - It produces severely incomplete analysis. Only use as an absolute last resort. On macOS, try the arm64 toolchain workaround or Rosetta first.
• "The build fails on macOS, just use build-mode=none" - Exit code 137 is caused by arm64e/arm64 mismatch, not a fundamental build failure. See macos-arm64e-workaround.md.
• "No findings means the code is secure" - Run check_db_quality.py and verify_query_suite.py and report that they passed. Without them, zero findings and a database that extracted nothing are the same output.
• "I'll just run the default suite" / "I'll just pass the pack names directly" - Each pack's defaultSuiteFile applies hidden filters and can produce zero results. Always use an explicit suite reference.
• "I'll put files in the current directory" - All generated files must go in $OUTPUT_DIR. Scattering files in the working directory makes cleanup impossible and risks overwriting

---

Content truncated.