erhe-headless-verify

erhe headless verification (headless Vulkan build + in-editor MCP)

The condensed run-book for the loop used to verify editor changes headlessly. Full reference: doc/agents/editor_runs.md.

Step 1 -- build

cmake --build build_vs2026_vulkan_headless --target editor --config Debug

If build_vs2026_vulkan_headless/ does not exist, configure once with scripts\configure_vs2026_vulkan_headless.bat. (The windowed ninja build scripts\build_ninja_win_vulkan.bat editor verifies compilation faster; capture_screenshot works in both builds since 2026-08-08, so headless mainly buys display independence.)

Step 2 -- launch and wait for the MCP server (NEVER blind-sleep)

FIRST kill any editor left over from a previous run. A stale editor.exe keeps port 3743 and SILENTLY eats every MCP call, so you drive an OLD binary and see impossible results (a fix that "does nothing", node counts that grow across runs) -- this wastes a lot of time. Then launch from the repo root (config/, res/, logs/ are cwd-relative) and poll logs/log.txt for the listening line (startup takes a variable few seconds; the port can fall back within [3743, 3763)):

Get-Process editor -ErrorAction SilentlyContinue | Stop-Process -Force   # no stale server on 3743
Start-Sleep -Milliseconds 800
if (Test-Path logs\log.txt) { Clear-Content logs\log.txt }
$p = Start-Process -FilePath "build_vs2026_vulkan_headless\bin\Debug\editor.exe" -WorkingDirectory (Get-Location) -PassThru -WindowStyle Hidden
for ($i = 0; $i -lt 60; $i++) {
    Start-Sleep -Milliseconds 1000
    if ((Test-Path logs\log.txt) -and (Select-String -Path logs\log.txt -Pattern "MCP server: listening" -Quiet)) { break }
}
Select-String -Path logs\log.txt -Pattern "MCP server: listening"   # "... on 127.0.0.1:3743 (pid <pid>, built <ts>)"

Then CONFIRM you are talking to the process you just launched (not a stale one):

py -3 scripts/mcp_call.py get_server_info    # { name, version, pid, build, port }

Assert the reported pid == $p.Id and build is your just-built binary; a mismatch, or a fallback port (3744+) in the listening line, means another editor.exe owns 3743. When that editor is a leftover of yours, kill it and relaunch; when it may be the user's own editor, leave it running and drive yours on the port from the listening line (ERHE_MCP_PORT=<port> or --port <port>). Remember $p.Id for cleanup.

Step 3 -- drive it with scripts/mcp_call.py

py -3 scripts/mcp_call.py --list                          # discover tools (--list <substring> filters)
py -3 scripts/mcp_call.py list_scenes
py -3 scripts/mcp_call.py render_scene_image b64:<base64-of-{"camera":{"eye":[0,2,5],"target":[0,0,0]}}>  # -> logs/render_scene_image.png
py -3 scripts/mcp_call.py capture_screenshot              # -> logs/mcp_screenshot.png
py -3 scripts/mcp_call.py get_scene_nodes b64:<base64-of-{"scene_name":"Default Scene"}>

• Pass tool arguments as b64:<base64 of the JSON object> (or - + stdin). Do NOT inline JSON with spaces on a PowerShell command line -- PowerShell 5.1 mangles the quotes. In PowerShell: $b64 = "b64:" + [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes($json)).
• Non-default port: --port <n> (from the listening line).
• Mutations (create_shape, geometry_graph_*, transform_selection, ...) take effect synchronously -- query tools immediately reflect them, and the geometry graph tools are undoable (drive undo / redo as tools, inspect with get_undo_redo_stack).
• Long evaluations: a mutation that triggers heavy main-thread work makes the server reply "Request timed out" (-32000) -- but the queued request STILL EXECUTES when the main thread gets to it ("Server busy: request queue full" requests are dropped instead). Retry queries freely; NEVER blindly re-issue a timed-out mutation (it double-executes and corrupts undo/redo counts) -- poll a cheap query until the server drains. Reference implementation: call() / mutate() in scripts/geometry_nodes_smoke_test.py.
• Geometry nodes regression sweep: py -3 scripts/geometry_nodes_smoke_test.py (65 checks; exit 0 = pass). Its incremental section needs config/editor/logging.json "editor.graph_editor": "trace" before launch -- revert before committing. Clean up res/editor/graphs/smoke_*.json after.
• Pick the image tool by what is verified. Scene content (rendering, lighting, GI, materials, A/B comparisons): render_scene_image renders the scene offscreen through an explicit camera (camera {eye, target, fov_y_degrees, ...} or camera_node) at an explicit width x height; no ImGui window, viewport size or overlay (grid, gizmo, selection outline) affects it, and output: "linear" gives HDR floats (.pfm) plus luminance stats. Editor UI (windows, widgets, gizmos, hover / selection feedback): capture_screenshot of the whole editor window. Details: doc/agents/mcp_server_usage.md "Image Tools". Neither can run inside batch.
• After render_scene_image / capture_screenshot, Read the PNG to actually see the frame. A useful trick to identify an object visually: select it and transform_selection it, then re-screenshot and diff by eye.
• Missing capability? Add a new MCP tool (handler + dispatch entry in src/editor/mcp/mcp_server*.{hpp,cpp}, schema in config/editor/mcp_tools.json) rather than working around it -- doc/agents/editor_runs.md calls this out as first-class debugging infrastructure.

Step 4 -- clean up (always)

Stop-Process -Id <pid> -Force
git status --short    # confirm the run dirtied no tracked file

An ERHE_AI_DRIVER=1 run writes no config file (doc/agents/editor_runs.md): the tracked config/editor/*.json files and the gitignored ImGui ini (config/editor/desktop_window_imgui_host_imgui.ini) are read but never written, so git status stays clean. Leave the ini alone.

Gotchas

• capture_screenshot works in BOTH builds (windowed since 2026-08-08: one-shot swapchain readback with a one-frame deferral inside the MCP server -- transparent to the caller). For deeper windowed GPU inspection (textures, pixel debugging) use the RenderDoc fork / erhe-renderdoc-gpu-debug.
• The windowed editor needs a live, awake display; headless does not.
• The scripted startup scene comes from config/editor/commands.json -- adjust it when a test needs a reproducible scene before init completes.
• One editor instance at a time. A second instance binds the next port in [3743, 3763), so your mcp_call.py (which defaults to 3743) keeps hitting the FIRST, stale one. Always kill all editors before launching (Step 2) and verify get_server_info reports the pid/build you expect. get_server_info + the (pid ..., built ...) suffix on the listening line exist precisely for this.