PoC: auth conn lost after a flap during AUTH_SYNC (fresh-create-community)

[holmesworcester]

Summary

This is a real bug in 7.1.0's production code (packages/backend/src/nest/qss/), not a flaky test or harness artefact.

A community owner finishes CREATE_COMMUNITY on QSS (qssSetup flips true on disk) and then the LFA auth handshake never completes — the create-community screen sits forever, even after the network steadies. The same code path runs for a member during invite redemption, so either side is exposed during the brief window between SIGN_IN_COMMUNITY and the LFA handshake completing.

The deterministic test in this PR fails roughly half the time against unmodified 7.1.0. The remaining halves where it passes confirm the underlying code path can succeed when timing happens to favour it — this is a race in the recovery path, not a hard wedge that needs a particular toxic to surface.

What the bug does (purely production code, no test logic involved)

When the QSS websocket disconnects mid-handshake:

1. QSSAuthConnection._onQssDisconnected (qss-auth-conn.ts:70-73) marks the conn INACTIVE.
2. QSSAuthConnectionManager._handleQssClientDisconnected (qss-auth-conn-manager.service.ts:70-73) calls close(false), which iterates the map and calls stopConnection (line 147-155) — this.authConnMap.delete(teamId). Every team's auth-conn entry is wiped from the map.

When the websocket reconnects:

3. QSSService._handleQssConnected (qss.service.ts:199-202) emits QSS_HANDLE_SIGN_IN once.
4. _handleQssHandleSignIn (qss.service.ts:217-254) acquires _signInMutex, sees qssSetup=true, takes the sign-in branch, calls _signInToCommunityImpl — which sends SIGN_IN_COMMUNITY with a 5 s ack timeout and then calls qssAuthConnManager.startNewConnection.

If that one sign-in attempt fails — either:

• (a) the SIGN_IN_COMMUNITY ack times out (5 s) under tail latency, or
• (b) the socket bounces between the if (!this.connected) check and the startNewConnection call —

then _signInToCommunityImpl returns ERROR with no retry queued. The websocket is connected (_scheduleReconnect doesn't fire), QSS_CONNECTED won't re-fire (we're already connected), so QSS_HANDLE_SIGN_IN won't either. No code path re-creates the auth conn. The user is stuck.

This is the same architectural shape as PRs #11 and #14: a single auto-flow attempt with no recovery path on a transient failure.

Failure trace from a failing run

+22439ms  startAuthConn-emitted    <teamId>     // qss.service.ts:777, end of _createCommunityImpl
+22439ms  auth-conn-set            <teamId>     // qss-auth-conn-manager.service.ts:137
+22445ms  startNewConnection-ok    <teamId>     // LFA handshake in flight
+22446ms  qss-disconnected         qssSetup=true; flap begins
+22450ms  auth-conn-deleted        <teamId>     // close(false) wipes the entry
+23220ms  qss-connected
+23234ms  startNewConnection-called             // sign-in branch re-runs
+23234ms  auth-conn-set
+23237ms  startNewConnection-ok
+23247ms  auth-conn-deleted                     // next flap-down deletes again
... (repeats per flap cycle) ...
+27000ms  qss-connected           (flap settles)
+27005ms  signIn-attempt-START
+32005ms  signIn-attempt-ERROR    SIGN_IN_COMMUNITY ack timed out
        ← nothing else fires; conn never re-created; test deadline elapses

What the test does

The test is a normal Jest spec that exercises the production code through real wire (Toxiproxy in the middle, real QSS server). It applies the seed=333 chaos profile, instruments the auth-conn lifecycle, then asserts qssAuthConnManager.getConnection(teamId) returns CONNECTED + JOINED within 60 s of the flap settling.

The test is not flaky in the "test bug" sense. It's deterministic in setup; the production code's behaviour under that setup is itself non-deterministic (the race window in step 4 above), which is why it fails ~50% rather than 100%. That's a property of the bug, not the test.

Suggested fix shapes

Two narrow options for the maintainer:

Option A — self-heal on _handleQssConnected. If qssSetup === true and qssAuthConnManager.getConnection(teamId) is undefined when QSS_CONNECTED fires (or shortly after), re-emit QSS_HANDLE_SIGN_IN after a short backoff so the sign-in retries.

Option B — retry-on-ERROR in _handleQssHandleSignIn. When _signInToCommunityImpl returns ERROR (and qssSetup is still true), schedule one more QSS_HANDLE_SIGN_IN emit after QSS_RECONNECT_DELAY_MS (or with exponential backoff). This generalises and would also paper over PR #11's stuck-state and PR #14's captcha-token-consumed issue.

Option B closes more bug surface for the same lines of code. Option A is a tighter, more local change. The maintainer should pick based on the codebase's invariants — this PR ships only the test, not the fix.

How to run the test

NODE_OPTIONS="--experimental-vm-modules" \
  ./node_modules/jest/bin/jest.js --config jest.stress.config.js \
  --runInBand --colors=false --forceExit \
  ./src/nest/qss/stress/scenarios/auth-conn-lost-after-flap.stress.spec.ts

Requires a local QSS server (npm run start:qss) and a Toxiproxy proxy (npm run start:qss:toxi). Node 20.20.1 for backend tests.