Skip to content

fix(backend): wait for in-flight account deletion retry before mongo stop - #3444

Merged
tyler-dane merged 2 commits into
mainfrom
fix/account-deletion-retry-shutdown-race-3442
Sep 7, 2026
Merged

fix(backend): wait for in-flight account deletion retry before mongo stop#3444
tyler-dane merged 2 commits into
mainfrom
fix/account-deletion-retry-shutdown-race-3442

Conversation

@claude

@claude claude Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Fixes #3442

What and why

PostHog error tracking caught a single staging occurrence of Could not retry pending account deletions did you forget to call \start`?, thrown from MongoService#accessInternalCollectionProps(mongo.service.ts:144) viaUserService.retryPendingAccountDeletions(user.service.ts:354, thependingSyncPurge` query).

Root cause: gracefulShutdown in app.ts calls userService.stopAccountDeletionRetries() (which only clearIntervals) before mongoService.stop(). clearInterval stops future firings, but a retry cycle already in flight from the previous interval tick keeps running as a detached void ...catch(...) promise. If shutdown lands mid-cycle, mongoService.stop() clears #internalClient while that cycle is still awaiting a pendingAccountDeletion query, so the next collection-getter access throws "did you forget to call start?". This matches the observed stack trace exactly (first query in the cycle succeeded, the second one at line 354 hit the now-undefined client) and the event only occurred on staging, consistent with a deploy/restart race rather than steady-state traffic.

Fix: track the in-flight retry cycle's promise on UserService and make stopAccountDeletionRetries async, awaiting that promise (each cycle already catches and logs its own errors) before returning. app.ts now awaits it in gracefulShutdown, so mongoService.stop() never runs while a retry cycle is still using the client.

Verify

684 pass, 1 skip, 0 fail, 1643 expect() calls. Ran 685 tests across 74 files.

Selected packages: backend
Checks run: test:backend, type-check, lint, knip
Checks skipped: (none)

VERDICT: PASS

New regression test (account deletion retry lifecycle in user.service.db.test.ts) fails on the pre-fix code (stopAccountDeletionRetries returned void, so nothing to .then() on) and passes with the fix.

Independent review

Self-reviewed only; this PR was opened by the error-autofix pipeline (mode: merge). No second human/AI reviewer yet.

Test plan

  • Added a regression test that reproduces the shutdown race (mocks retryPendingAccountDeletions to stay in flight, starts the retry timer, and asserts stopAccountDeletionRetries() doesn't resolve until the cycle finishes)
  • Verified the new test fails against the pre-fix user.service.ts/app.ts and passes with the fix
  • bun run verify --strict is green

…stop

Graceful shutdown cleared the retry interval but never awaited a cycle
already in flight, so mongoService.stop() could tear down the client
mid-query and the interval callback's pendingAccountDeletion access
threw "did you forget to call \`start\`?" (PostHog issue, staging,
single occurrence).
@claude claude Bot added autofix Opened/handled by the error-autofix pipeline automerge-candidate Autofix agent judged this fix confident enough for auto-merge (still gated by a deterministic check) labels Sep 6, 2026
@claude
claude Bot marked this pull request as ready for review September 6, 2026 00:56
Co-authored-by: Tyler Dane <tyler-dane@users.noreply.github.com>
@cursor

cursor Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Still relevant: the shutdown race is present on current main (stopAccountDeletionRetries still only clearIntervals, and gracefulShutdown does not await it). Merged latest main into this branch so the PR is up to date. Re-running bun run verify --strict next.

@tyler-dane
tyler-dane enabled auto-merge September 7, 2026 20:02
@cursor

cursor Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

bun run verify --strict after merging latest main:

Selected packages: backend
Checks run: test:backend, type-check, lint, knip
Checks skipped: (none)

✓ All checks passed
VERDICT: PASS

The new regression test (UserService > account deletion retry lifecycle) passed. Enabled merge (repo merge queue / squash).

@tyler-dane
tyler-dane added this pull request to the merge queue Sep 7, 2026
Merged via the queue into main with commit e0aa7a2 Sep 7, 2026
15 checks passed
@tyler-dane
tyler-dane deleted the fix/account-deletion-retry-shutdown-race-3442 branch September 7, 2026 20:06
@github-actions github-actions Bot mentioned this pull request Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autofix Opened/handled by the error-autofix pipeline automerge-candidate Autofix agent judged this fix confident enough for auto-merge (still gated by a deterministic check)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error

2 participants