Skip to content

Commit 39fa32e

Browse files
committed
Quiesce every Python worker path before deregistration
1 parent a6a1f98 commit 39fa32e

4 files changed

Lines changed: 574 additions & 74 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,10 @@ Earlier SDK versions remain historical releases and are not alternate supported
5151
`NamespaceDescription.deleted`.
5252

5353
### Fixed
54+
- Worker shutdown now quiesces inline `run_until()` work and fully stops the
55+
query-poller thread before deregistration. A worker that cannot stop every
56+
execution path by its shutdown deadline reports a failure and keeps its
57+
registration instead of allowing stale completions after lease recovery.
5458
- Workers configured with only `worker_token` can now authenticate cluster
5559
discovery and complete registration, polling, heartbeat, and graceful
5660
deregistration without receiving a control credential. Actual worker and

‎docs/reference/worker.md‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,19 @@
33
## Lifecycle and shutdown
44

55
`Worker.run()` and `Worker.run_until()` register through the worker protocol
6-
before polling. After registration succeeds, shutdown stops the pollers, waits
7-
up to `shutdown_timeout` for in-flight tasks, and then removes that worker-plane
8-
registration. Calling `stop()` more than once is safe and sends at most one
6+
before polling. After registration succeeds, shutdown stops every poller and
7+
waits for accepted work within the shared `shutdown_timeout` deadline before
8+
removing that worker-plane registration. Work still running at the deadline is
9+
cancelled. Calling `stop()` more than once is safe and sends at most one
910
deregistration request. No deregistration is sent if registration failed.
1011

12+
An external `stop()` also interrupts `run_until()`; the pending `run_until()`
13+
call ends with `asyncio.CancelledError` after its accepted workflow or activity
14+
work has drained or been cancelled. If an async poller, the query-poller thread,
15+
or accepted work cannot be stopped, `stop()` raises `RuntimeError` and leaves
16+
the registration active. Treat that as an incomplete shutdown requiring
17+
operator attention rather than continuing as though the worker stopped cleanly.
18+
1119
For a long-running process, coordinate shutdown with the same worker instance
1220
and await both `stop()` and the run task:
1321

0 commit comments

Comments
 (0)