online-multiplayer-debug

Online Multiplayer Debug Harness

Overview

Mission Geo's online module (casual online rooms + ranked matches) has a history of regressions that only show up with two real clients exchanging real Firestore/RTDB state — a long wait between rounds, a player who queues then disconnects and freezes the pairing, a host that stalls forever on roundResult. None of these reproduce with a single dev-flavor instance or the security-rules emulator suite (online-security-validation — that's for rule/transaction correctness, not client-visible pacing or real 2-device disconnect races).

This harness boots 2 real Android emulators, installs the same accelerated dev-debug build on both, and drives them as 2 real players against the real mission-geo-dev Firebase project (never prod — dev flavor only). An optional timing profile (DEBUG_ONLINE_SPEED / DEBUG_RANKED_SPEED dart-defines) compresses match PACING only, so a full casual match or ranked round completes in well under a minute instead of several.

This is the validation gate for every online-touching change — casual rooms, ranked matches, presence/heartbeat, Cloud Functions under functions/ that touch game_rooms / ranked queues. Don't claim an online fix is verified without walking at least the casual + ranked self-proofs below, plus whichever chaos scenario the change targets.

When to Use

• Any change to lib/providers/online_game_provider.dart, lib/providers/ranked_provider.dart, lib/providers/ranked_match_provider.dart
• Any change to lib/core/services/firebase/firebase_game_room_service.dart or firebase_ranked_service.dart
• Any change to a Cloud Function touching game_rooms, ranked queues, or presence (functions/index.js)
• Any change to lib/pages/modes/online/** or lib/pages/modes/ranked/**
• Before merging a PR that touches the online module (in addition to, not instead of, online-security-validation when rules/transactions are involved)

Skip for: pure défi/adventure/duel-local changes with no online/ranked surface.

Prerequisites

• 2 free ports on the shared emulator pool (.claude/skills/shared/emulator-pool.sh — 5554/5556/5558). Check with mg_emu_status before claiming; this harness needs 2, so don't start it if another agent already holds 2+ ports.
• flutter on PATH (export PATH="$HOME/flutter/bin:$PATH" on this machine).
• The dev Firebase project (mission-geo-dev) reachable — this is the REAL backend, not an emulator. Both instances write real data (rooms, ranked stats, queue entries) to it. Debug-created rooms are cheap and expire (purgeExpiredRooms), ranked debug-solo rooms are prefixed DBG, but a real 2-player ranked match DOES affect real ELO/stats for whatever dev account each emulator is signed in as.
• Two DISTINCT Firebase identities, one per instance. Naively booting mg_emu_up phone phone would launch two QEMU processes of the SAME golden mission_geo_phone.avd image — and since adb install -r preserves existing app data, both instances would inherit the identical baked-in anonymous UID (whatever was last signed in on that AVD from ordinary single-instance dev usage), silently invalidating every 2-player scenario (join by code, ranked pairing, all 5 chaos scenarios).
  • Fix in place: online-duo-up.sh auto-provisions a SECOND, persistently distinct AVD (mission_geo_phone_b, a one-time clone of mission_geo_phone decoupled via a single non-read-only -wipe-data boot — see setup-phone-b-identity.sh) and boots instance A on mission_geo_phone / instance B on mission_geo_phone_b. Both keep stable, distinct identities across every future run — no re-provisioning, no re-onboarding. First run costs a few minutes (disk clone ~3-4GB + one decoupling boot cycle); every run after is instant (idempotent, sentinel-gated).
  • Automated guard: regardless of the above, online-duo-up.sh verifies the two instances actually ended up with distinct UIDs after boot (mg_chaos_tail_uid on both, fails loudly with a non-zero exit on a match) — see "MG_DEBUG_UID identical on both instances" below if it fires. Don't remove this guard even though the fix above should make it a no-op in the common case: it's what stops this exact bug from silently coming back as a false PASS.
  • Only the default --avd phone path gets the auto-provisioned decoupled clone; overriding --avd to tablet7/tablet10 boots the same tablet AVD for both instances and the guard is expected to fail unless you provision your own distinct identity for it first.
  • Onboarding/consent re-appearing on mission_geo_phone_b every boot? Every harness boot of it is -read-only (see setup-phone-b-identity.sh's header comment), so anything written to SharedPreferences during a read-only session — including "onboarding done" / consent choice — is discarded at exit. If mission_geo_phone_b keeps landing on the onboarding/consent screen instead of home on every online-duo-up.sh run, persist it the same way the identity itself was decoupled: one non-read-only boot to click through onboarding + consent once, then a clean shutdown so it's written back to the AVD's own base image, exactly the pattern setup-phone-b-identity.sh already uses for the fresh sign-in. Concretely: claim a port, boot mission_geo_phone_b WITHOUT -read-only (no -wipe-data this time — you want to keep the existing decoupled identity, just add persisted prefs on top), install/launch, manually tap through onboarding + accept/decline consent once, then adb -s emulator-<port> emu kill for a clean shutdown (not mg_release_port's zombie-reap path — a normal kill on a non-read-only session flushes to disk same as the identity-decoupling boot does). Every subsequent read-only boot then starts on home, same as mission_geo_phone already does from ordinary dev usage.
• App Check debug token file, per worktree. android/firebase_debug_token.dev.properties is gitignored (see CLAUDE.md's "App Check (debug token pinned)" section), so every fresh worktree (git-workflow-branch-worktree skill) starts without it — BuildConfig.FIREBASE_APP_CHECK_DEBUG_TOKEN is then blank, MainActivity.installFixedAppCheckDebugTokenIfNeeded() no-ops, and the App Check SDK falls back to a fresh random UUID on every install (not allow-listed in the mission-geo-dev Firebase Console), silently taking down every Firestore/RTDB call this harness depends on with PERMISSION_DENIED. Fix in place: online-duo-up.sh checks for the file before doing anything else and copies it from the primary checkout (/home/mrjack/git/mission-geo/android/firebase_debug_token.dev.properties) with a loud log line, hard-failing with a clear message only if neither copy exists. See "App Check: random UUID / PERMISSION_DENIED storm" below if it still fires.
• App Check pm-clear race (a flutter build/reinstall inside online-duo-up.sh never triggers this — mg_emu_install uses adb install -r -d, which preserves app data; this only bites a genuinely fresh app-data state: first-ever install on a golden AVD image, or an explicit pm clear). On a fully fresh data state, MainActivity's SharedPreferences write can lose the race against the App Check SDK's own lazy token generation — 3 different random tokens were observed across 3 cycles in practice. See "App Check: random UUID / PERMISSION_DENIED storm" below for the reliable workaround (write the SharedPreferences XML yourself via adb shell run-as, before first launch).

Quickstart

cd /home/mrjack/git/mission-geo   # or your worktree
export PATH="$HOME/flutter/bin:$PATH"
bash .claude/skills/online-multiplayer-debug/scripts/online-duo-up.sh

This claims 2 pool ports, boots mission_geo_phone (instance A) + mission_geo_phone_b (instance B — a persistently distinct clone, auto-provisioned on first run only, see "Two DISTINCT Firebase identities" above), builds ONE accelerated dev-debug APK (DEBUG_ONLINE_SPEED=6, DEBUG_RANKED_SPEED=3 by default — override with --online-speed N / --ranked-speed N), installs it on both, and force-foregrounds the app. It prints the two ANDROID_SERIAL values (e.g. emulator-5554 / emulator-5556) — keep them, every chaos primitive below needs an explicit serial. First run only takes a few extra minutes to provision mission_geo_phone_b (disk clone + one decoupling boot) — every run after is fast.

Both instances MUST run the identical build. The pacing scalers are compiled-in consts (lib/core/dev/debug_timing_config.dart) — a speed mismatch between A and B desyncs their countdowns/reveals. The script builds once and installs the same APK on both; don't flutter run a second, differently-configured instance against one from this script.

Iterating on non-harness online code: rerun online-duo-up.sh (drop --skip-build only if you didn't touch pacing/debug-tile files — otherwise you're testing a stale build).

Persistent adb attachment (survives a session handoff). This harness never runs flutter run — it builds once, adb installs, then drives everything with one-shot adb/logcat calls — so the only thing keeping a claimed port's lease fresh is mg_emu_up's mg_keepalive_start daemon, and that daemon dies the instant ITS OWNING SESSION dies (by design — a dead agent must not pin a port forever). That's a problem for the common pattern of one agent booting the harness and then handing off to another (e.g. an inspector agent that only pokes at the emulators with one-shot adb): if the booting agent's session ends first, its keepalive dies, and the pool's own per-port keeper reclaims the emulator once the lease goes stale (MG_LEASE_TTL, 180s by default) — even though a different agent is still actively using it. online-duo-up.sh now arms one adb -s <serial> logcat per claimed serial right after boot (mg_emu_arm_logcat_pin in mg-emu.sh) specifically to survive this: the pool's attachment checks

---

Content truncated.