test-script

Test Scripts

Overview

Write test scripts for git-spice commands as .txt files in txtar format. Test scripts verify end-to-end behavior and are stored in testdata/script/.

Quick Reference

Task	Command/Pattern
Run specific test	mise run test:script --run $name
Update golden files	mise run test:script --run $name --update
Create branch	gs branch create <name> -m <msg>
Make commit	gs commit create -m <msg>
Compare stdout	cmp stdout $WORK/golden/file.txt
Partial match	stdout 'expected substring' for non-JSON text only
Check file exists	exists filename
Compare with env vars	cmpenv stdout $WORK/golden/file.txt
Test interactive	env ROBOT_INPUT=$WORK/robot.golden ROBOT_OUTPUT=$WORK/robot.actual

Common Mistakes

Avoid these errors when writing test scripts:

• ❌ Using shell operations: Test scripts don't support >, >>, 2>, |, or command substitution. Use cmp stdout, cmp stderr, stdout, and stderr instead.

• ❌ Using cat or ls: Use exists filename to check existence, cmp have_file want_file to compare contents.

• ❌ Forgetting as and at: Always freeze time and author for deterministic Git hashes.

• ❌ Using git checkout -b: Use gs branch create instead (unless testing non-git-spice scenarios).

• ❌ Empty golden files with (empty): Just leave content blank after -- path --.

• ❌ String matching JSON output: Use cmpenvJSON for ShamHub JSON dumps and other JSON outputs. Do not use stdout or stderr substring assertions on JSON.

Writing a test script

Test scripts are plain text files with a series of commands. Most test scripts follow this general structure:

# Brief description of what the test does.
# Wrap comments at 80 characters.

as 'Test <[email protected]>'
at '2025-10-18T21:28:29Z'

# Setup
cd repo
git init
git commit --allow-empty -m 'Initial commit'
gs repo init

# (Optional) ShamHub setup for forge interactions
shamhub-setup
shamhub register alice
shamhub new origin alice/example.git
git push origin main

# Login as user
env SHAMHUB_USERNAME=alice
gs auth login

# ...

some command
cmp stdout $WORK/golden/output.txt

-- repo/file.txt --
file inside repo
-- repo/path/to/another-file.txt --
another file inside repo
-- extra/supporting-file.txt --
supporting file outside repo.
may be copied into repo during test.
-- golden/output.txt --
expected output for comparison

Notes:

• Deterministic Git hashes

  To get Git SHAs that are consistent across test runs, always freeze time and author at the start of the script using the at and as commands.

  as 'Test <[email protected]>'
  at '2025-10-18T21:28:29Z'
  

  The at should always use the current time at the moment of writing the test script.

• Repository initialization

  Initialize the Git repository and git-spice within it:

  cd repo
  git init
  git commit --allow-empty -m 'Initial commit'
  gs repo init
  

• ShamHub setup

  For tests involving forge interactions, we use a simulation server called ShamHub. To set it up, use shamhub-setup, then use shamhub commands to create users and repositories:

  shamhub-setup
  shamhub register alice
  shamhub new origin alice/example.git
  git push origin main
  
  env SHAMHUB_USERNAME=alice
  gs auth login
  

Following setup, use git-spice via gs commands to perform actions.

Test script style

• Naming

  Follow the convention: <command>_<scenario>.txt Where <command> is the git-spice command being tested in snake_case, and <scenario> describes the specific test case. Examples:

  • repo_sync_conflict.txt
  • stack_submit_with_labels.txt

  If the test script is a regression test for a specific issue, include the issue number in the name:

  • issue123_stack_rebase_does_not_do_a_thing.txt

• Comments

  Use comments (#) to explain the purpose of each step. Wrap comments at 80 characters for readability.

  If the test script is a regression test for a specific issue, include a comment with the issue link at the top.

Common tasks

Creating branches

gs branch create <branch-name> -m <commit-msg>

This will create a new branch and commit all staged changes to it with the given commit message.

See gs branch create --help for more options.

Never use manual git checkout -b unless specifically testing a non-git-spice branch creation scenario.

Making commits

gs commit create -m <commit-msg>

This will commit all staged changes to the current branch with the given commit message.

See gs commit create --help for more options.

Supporting files

Files needed by test scripts are defined at the end of the script using -- path/to/file -- syntax.

For each such section, everything after the -- path/to/file -- line until the next -- path/to/another/file -- line, or the end of the file, is treated as the contents of that file.

-- repo/feature.txt --
Initial content of feature.txt
-- extra/feature-new.txt --
Updated content of feature.txt
-- golden/output.txt --
Expected output of the command

Using golden files

If the expected output of any command is stored in a supporting file, put it inside a golden/ subdirectory in the test script, and refer to it using $WORK/golden/filename.

Tip: For outputs that we do not control (e.g. git commit hashes), save time by putting placeholder values in the golden files, then run the test script with the --update flag to auto-update them. (This only works for cmp comparisons, not cmpenv or cmpenvJSON.)

Verifying git state

• View git graph:

  git graph --branches
  cmp stdout $WORK/golden/graph.txt
  

• View git status:

  git status --porcelain
  cmp stdout $WORK/golden/status.txt
  

• Verify git-spice branch relationships:

  gs ls
  cmp stderr $WORK/golden/ls.txt
  

• Verify git-spice branch and commit details:

  gs ll
  cmp stderr $WORK/golden/ll.txt
  

Verifying command output

• Matching entire stdout

  run some command
  cmp stdout $WORK/golden/output.txt
  

• Matching stdout partially

  run some command
  stdout 'expected output substring'
  

• Matching entire stderr

  run some command
  cmp stderr $WORK/golden/error.txt
  

• Matching stderr partially

  run some command
  stderr 'expected error message substring'
  

• Comparing with environment variable substitutions

  run some command
  cmpenv stdout $WORK/golden/output.txt
  cmpenv stderr $WORK/golden/error.txt
  

  ($FOO placeholders in the golden file will be replaced with their actual values from the test environment.)

• Comparing JSON output with environment variable substitutions

  run some command --json
  cmpenvJSON stdout $WORK/golden/output.json
  

  ($FOO placeholders in the golden JSON file will be replaced with their actual values from the test environment.)

  For ShamHub JSON dump fixtures, start with a plain cmp assertion, run the script with --update, and then replace volatile ShamHub URLs such as http://127.0.0.1:<port> with $SHAMHUB_URL in the generated golden file. After that, change the assertion to cmpenvJSON.

Testing Interactive Prompts

For commands with interactive prompts,

• Declare a golden fixture file with pre-defined answers for each expected prompt. These are JSON-encoded answers separated by === lines, with > comments that contain the rendered prompt text.

  -- robot.golden --
  > Enter your name:
  "Alice"
  ===
  > Choose an option:
  >  1) Option A
  >  2) Option B
  2
  

• Before invoking the interactive command, set the ROBOT_INPUT environment variable to point to the fixture file, and ROBOT_OUTPUT to capture rendered prompts.

  env ROBOT_INPUT=$WORK/robot.golden ROBOT_OUTPUT=$WORK/robot.actual
  

• Run the command that prompts for input.

  gs some interactive command
  

• After the command completes, compare the captured output file with the golden fixture file.

  cmp $WORK/robot.actual $WORK/robot.golden
  

  (Use cmpenv if environment variable substitutions are needed.)

Testing opening browser URLs

For commands that open browser URLs, set BROWSER_RECORDER_FILE to a file path that will capture the URLs that were requested to be opened.

env BROWSER_RECORDER_FILE=$WORK/browser.log
gs branch submit --web

Verify the recorded URLs against the golden copy:

cmpenv $WORK/browser.log $WORK/golden/browser.log

Running test scripts

• Run a specific test script:

  mise run test:script --run $name
  

  Where $name is the script name without .txt extension. For example, to run testdata/script/repo_sync_conflict.txt:

  mise run test:script --run repo_sync_conflict
  

• Auto-update golden files:

  If a test script fails because the expected output has changed intentionally, use the --update flag to update golden files:

  mise run test:script --run $name --update
  

  Note: This only works for cmp comparisons. cmpenv and cmpenvJSON do not support --update mode.

ShamHub

ShamHub is a simulated forge server (e.g., GitHub/GitLab) used for testing git-spice's forge interactions.

Outside of using git-spice (gs) commands, ShamHub interactions are available through the shamhub command defined at internal/forge/shamhub/cli_main.go.

Common shamhub commands:

• shamhub-setup: Initialize the ShamHub server and set environment variables.
• shamhub register <username>: Register user

---

Content truncated.