Repository navigation
fix(server): tool calls a released session cuts off no longer keep the thread Waiting - #29036
NotReliable wants to merge 8 commits into
Conversation
…e thread Waiting When T3 releases a provider session whose background work ends with its process, such as the detach after t3_worktree_handoff or a provider switch, the session cannot report how its open tool calls ended. Run finalization keeps command and dynamic tool items open for a late result, so these items stayed running. The thread then showed "Waiting on background task" until a restart. The release path already clears the dead session's background-task roster. It now also marks that session's open command_execution and dynamic_tool items as interrupted, as Stop does for items on a dead session. It reads the item ids when the session leaves the live map, so items of a replacement session stay open. Runtimes whose tasks can outlive them, and server shutdown, keep their items as before. Fixes pingdotgg#15993 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review repairs: - Select the cut-off items at write time: open tool calls on the released provider threads that started at or before the release. This includes an item that the dead session's queued events stored after the release began. Items that a replacement session started later stay open. - Write the interrupt only if each item row is unchanged (EventSink.writeIfTurnItemsUnchanged), with three tries like the roster. A late completion from the dead session now wins over the interrupt. - Use Effect.fn spans for both release writers, and set driver on the events. - State in the backgroundTasksEndWithProcess comment that the flag also covers tool calls. - Extend the existing roster tests instead of adding new ones, add the server shutdown row, and keep one integration case (the handoff). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l write Review repair: one conditional write for all cut-off items let three late completions use up the three tries, and the other open items stayed running. Each item now gets its own conditional write. A late completion rejects only the interrupt of its own item, and that item is then no longer open. Items that are still open after a conflict, such as after a late output update, are read again and tried again, up to three times. EventSink.writeIfTurnItemUnchanged replaces the batch write, and both conditional writes share one helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review repair: the retry read only the items that conflicted. A conflict shows that the dead session's queued events are still landing, so they can also store more cut-off items. The retry now reads all open cut-off items again. Items that the earlier pass interrupted are no longer open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review repair: a pass with no conflict returned at once, so an item that the dead session's queued events stored during that pass was never read. Each pass now ends with a new read, and cleanup stops only when that read finds no open cut-off item, or after three passes. The warning then names the count of open tool calls. The raced test checks statuses by item id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review nit: the raced test stores one more cut-off call during the third pass and checks that it stays open, so the pass bound is tested. The comment on the late calls now says that only the first four writes complete one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ApprovabilityVerdict: Approved at Macroscope's review found this PR approvable — Released sessions now mark their cut-off tool calls interrupted instead of leaving threads Waiting. Cleanup is adapter-gated and uses transactional conditional writes to protect concurrent updates. You can add or adjust custom eligibility rules. Learn more. |
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
|
Note Written by merged current main without rebasing, preserving both sides of the test-adapter conflict. fixed a release race where queued provider events could reopen a tool call after cleanup. released subscriptions now clean up once on close, including drain, runtime failure, graceful provider stop, and early consumer exit. late completions and replacement-session work keep their existing protections. blacksmith verification on Verified in a real dev environmentverified unchanged head
main after recovery, with the orphan still running: pr after the same release and recovery, with no stale running command: Not verifiedthe exact race where the handoff tool itself loses its result: ordinary real handoffs on both revisions delivered that result before detach. the command cutoff and dynamic-tool cutoff paths were exercised separately through the running server. the handoff run still ends with a provider-stream failure, as explicitly excluded from this pr. muse, native desktop/mobile clients, and relay/tunnel modes were not exercised. cleanup contention and the three-pass bound were covered by existing tests, not induced in the live app. |
|
Still reproduces on |



Problem
After
t3_worktree_handoff, the thread shows "Waiting on background task mcp__t3-code__t3_worktree_handoff" for good, also after later turns complete (#15993). The handoff'sdynamic_toolitem staysrunningwithcompletedAt: null.The handoff detaches the Claude session.
releaseEntryfails the run's subscribers before the adapter can report how its open tool calls ended. Run finalization keepscommand_executionanddynamic_toolitems open for a late result, so nothing ever ends them.pendingBackgroundTurnItemsthen lists the item, and the thread waits. The same leftover occurs on any release of such a session while a tool call runs, for example a provider switch.Seen on
0.1.3-nightly.20261026.3717(Linux,claudeAgent): in one state database, 6 of 6 handoff tool items wererunninginside afailedrun.Change
The release path already clears the dead session's background-task roster ("Dead processes cannot report task endings"). It now also interrupts the session's cut-off tool calls, with the same gate.
releaseEntry(session) readReleasedProviderThreads # gate: backgroundTasksEndWithProcess, not server_shutdown failSubscribers closeScope # waits for the process to stop writeReleaseRecords writeReleasedRosterEvents # existing: clear the task roster + writeReleasedToolCallEvents # new: open command_execution / dynamic_tool items + repeat up to 3 passes + read open tool calls on the released provider threads with startedAt <= releasedAt + stop when none is open + for each: writeIfTurnItemUnchanged(item, status: "interrupted")command_executionanddynamic_toolitems on the session's provider threads that started at or before the release. Items that a replacement session starts later stay open. Subagent rows are not changed, because run finalization already ends run-owned ones.interrupted, the status Stop already gives items on a dead session.EventSink.writeIfTurnItemUnchanged(same pattern aswriteIfProviderThreadUnchanged, now one shared helper) commits only if the stored item is unchanged. A completion from the dead session's queued events rejects only the interrupt of its own item. Each pass ends with a new read, so items that queued events store during cleanup are also closed.backgroundTasksEndWithProcess(Claude and Muse; Muse's host runs all its session work). Runtimes whose work can outlive the session, andserver_shutdown, keep their items as before. The flag's comment now says that it covers tool calls too.Door: two-way. No schema, contract, or setting change; the only new API is one internal
EventSinkmethod.Revert: revert the PR. Items that it already marked
interruptedstayinterrupted, the same state that Stop writes today.Blast radius: server-only. Every release of a Claude or Muse session (detach, switch, idle release, close) now does one or two more filtered turn-item reads per released provider thread.
Scope and approval
Fixes #15993 (triaged
bug). The maintainer triage there names this direction: publish the tool cancellations on an exclusive-session release.This addresses the objection that closed #17306 ("marks tool calls interrupted after ingestion errors without stopping the provider"). The change does not touch the ingestion-failure path. It runs only in
releaseEntry, after the session scope close has waited for the provider process to stop, so the provider loss is confirmed. The integration harness follows the one in #17306.Out of scope, by choice:
failed, and the continuation behavior is unchanged ([Bug]: Claude t3_worktree_handoff still fails the turn and its continuationPrompt never starts #22971; fix(server): successful worktree handoffs complete normally #24545 and fix(server): Claude child continues after a worktree handoff #22825 work on it).backgroundTasksEndWithProcess.Verification
Red first. The new assertions fail on
mainand pass with the change:maintestkit/ProviderSwitch.integration› interrupts the handoff tool call that releases its own session (realthread.metadata.update→ detach, assertsderivePendingBackgroundWorkis[])expected 'running' to equal 'interrupted'ProviderSessionManager› clears the background work of a session a detach releases (cut-off →interrupted; completed, other-session and replacement items unchanged)ProviderSessionManager› keeps provider updates that land while release clears (4 late completions win, a late output update and items stored during cleanup are still interrupted, the 3-pass bound holds)ProviderSessionManager› keeps the background work after the release of a runtime whose tasks can outlive it / server shutdowncancelledstatus, noserver_shutdowngate.vp run check:changed: format, lint, knip, graphql, typecheck (25 packages) and the tests beside the changed files all pass. 9 affected test files (ProviderSessionManager, ProviderSwitch.integration, BackgroundWorkStop.integration, SelectionRestart.integration, RunExecutionService, ProviderRuntimeRecoveryService, WorktreeMcpService, FoundationPersistence, ProviderEventIngestor): 275 tests pass.Not checked:
provider-session-release-tool-call-conflictwith the count.Skills: core-v2.0.0
Model and harness: Claude Opus 5.5 through Claude Code in T3 Code.
🤖 Generated with Claude Code