Exp 285: the worker that never sleeps

[danReynolds]

Hypothesis

Yesterday's exp 284 stopped nine experiments in their tracks by doing something
none of them had done: measuring the write hop before proposing a mechanism for
it. Of the 6.28 µs between await db.execute() and the identical insert run
inline on the calling isolate, 5.30 µs turned out to be the isolate boundary
and only 0.98 µs was everything resqlite actually does (claim 284.1). Its
rejection came with a condition for reopening the family — a candidate that
attacks the boundary rather than the bookkeeping.

Inside that boundary it left one number sitting there unexplained. The very
same executeWrite call — same C entry point, same statement, same parameters,
same handle — costs 7.612 µs on a running isolate and 9.380 µs on a worker that
was parked until the message arrived (claim 284.3). About 1.8 µs of every write
is just the woken side running cold.

So this run challenges an assumption nobody in the program has questioned:
that an idle worker isolate should idle. Every resqlite worker handles its
message, returns to its event loop, and lets the VM park its thread. If that
1.8 µs is the VM releasing and re-acquiring the thread, then a worker that never
runs out of work never pays it — and the boundary is partly a scheduling
artifact rather than a transport cost. Unlike exps 271 and 279, which attacked
the same boundary from the completion side and the transport side, this changes
neither what rides on the message nor who runs SQLite. It changes only what
state the receiver is in when the message lands.

Stated up front as the risk budget: a worker that stays runnable burns a core,
which is a real cost in a library that ships to phones. So a win had to be
priced in CPU as well as wall time.

Approach

Dart gives a library exactly one lever here — leave something in the isolate's
event loop — and microtasks are disqualified before you write any code, because
a microtask loop runs to exhaustion before the event loop delivers a port
message, so the isolate would never receive the request it is staying awake for.
That leaves two primitives, and the harness measures both.

benchmark/experiments/worker_keep_warm.dart extends exp 284's floor writer —
a hand-rolled isolate writer with none of resqlite's machinery, so the isolate
architecture is priced on its own — with four lanes: cold (the worker as
exp 284 left it), timer (a self-rescheduling zero-duration Timer),
self (the worker sends a token to its own receive port, so tokens and
requests share one FIFO queue), and both (self on the worker and the
same loop on the calling isolate, which parks awaiting the reply exactly as the
worker parks awaiting the request). Each worker times its own executeWrite
calls, so the cold tax is read inside the C call rather than inferred from a
round trip, and process CPU is read through getrusage around every block.

The shippable half was then built into resqlite's real writer isolate and A/B'd
through await db.execute(...), two Database handles alternating in one
process. Full detail in
experiments/285-worker-keep-warm.md.

Results

The cold tax is real and recoverable. The identical C call, medians of three
passes over 6,400 writes per worker:

worker	executeWrite µs	Δ
cold	9.606	—
self	8.974	−6.6%
self (inside both)	8.724	−9.2%

Recovering it costs more than it is worth.

lane	µs/write	Δ vs cold	CPU µs/write	Δ CPU
cold	12.970	—	14.76	—
timer	16.218	+25.0%	23.35	+58%
self	13.710	+5.7%	16.69	+13%
both	12.078	−6.9%	21.80	+48%

In resqlite's real writer the loss is wider still: +9.6% at a 15 µs window,
+9.0% at 40 µs, +6.2% in a third collection, CPU up 13–17%.

So the mechanism does what it was supposed to do and still loses. Keeping the
worker runnable really does buy back 0.6–0.9 µs inside SQLite, but a request
that arrives mid-spin waits behind a token turn — and a self-sent port message
turn costs 1.85 µs, which is more than the residency it buys. The timer lane
loses far worse because its turns cost 6.50 µs each. A control at a 5 µs window,
where the round trip outlasts the window so the loop exits on its first turn and
never engages (spins=0), reads flat at 14.567 vs 14.523 µs — confirming both
the tax and the cost come from the loop actually running.

The lane that wins is the one resqlite cannot have. both is the only lane
of four that beats cold, and against the inline reference it cuts the isolate
boundary from 5.117 µs to 4.225 µs — 17% of what claim 284.1 prices at
5.30 µs. The boundary really is substantially scheduling rather than
transport. But the half that wins belongs to the caller, and the caller in a
Flutter app is the UI isolate; a token loop in its event loop is exactly the
main-isolate blocking exp 280 rejected a candidate for (claim 280.2). There is
no version of this that ships.

And that yields the finding most worth carrying forward. cold's caller
parks between writes only because the benchmark gives it nothing else to do. The
release suite's Single Inserts (100 sequential) row — the one public lane where
resqlite loses to a peer, 15.8 µs against raw sqlite3's 9.3 µs — awaits each
write and idles in between, so about a sixth of the boundary it charges resqlite
is the caller's own wake, which an application issuing the same hundred writes
from an isolate that has frames and futures pending never pays. The row is a
fair worst case. It is not the typical one, and it should be cited that way.

Outcome

Rejected, as a moonshot that confirmed its mechanism and closed its
direction anyway. Worker-side keep-warm — the only half a database library
controls — is 6–10% slower and 13–17% more expensive in CPU in the shipping
path, reproduced across three windows and nine passes.

Would reopen if the VM offers a way to keep an isolate's thread hot without
keeping the isolate runnable: a spin-before-park window in the isolate
scheduler, or thread affinity for a long-lived isolate. That is a Dart SDK-shaped
change, not a library-shaped one, and this run is the evidence for what it would
be worth — roughly 0.9 µs per request on the worker side, plus the caller's wake
for an application that is genuinely idle.

Do not reopen by making the window adaptive, the token cheaper, or the loop
smarter. The sweep already covers 5 µs to 200 µs and the shape is flat: self
is above cold at every window. The self-sent port message is already the
cheapest keep-alive Dart exposes, and it costs twice the residency it buys back.

Runtime reverted — lib/ on this branch is byte-identical to main. Prototype
and the shipping-path lane at archive/exp-285; the four floor lanes retained
as benchmark/experiments/worker_keep_warm.dart, whose --part=both lane is
the durable instrument: it separates the two thread wakes in a round trip and
shows how much of a benchmark's hop is the benchmark's own idle caller.

Test plan

• dart analyze --fatal-infos lib benchmark/experiments/worker_keep_warm.dart — clean
• dart test test/database_test.dart test/reader_pool_test.dart test/transaction_test.dart test/experiment_outcomes_test.dart test/knowledge_impact_test.dart test/knowledge_pins_test.dart — 198 passing
• dart test test/benchmark_generated_outputs_test.dart test/tracelite_workflow_docs_test.dart — passing
• dart run benchmark/finalize_experiment.dart --experiment=experiments/285-worker-keep-warm.md — green
• dart run tool/knowledge/impact.dart origin/main — five new claims, nothing retired
• git diff origin/main -- lib/ empty; retained harness rebuilt and re-run on the reverted tree, reproducing the pattern a fourth time
• Benchmark Run: none — rejected experiment, no runtime code ships (lib/, native/, hook/ unchanged)

🤖 Generated with Claude Code

[github-actions]

Belief impact

Learned

• 285.1 · The worker that never sleeps
  The cold tax on a woken worker is thread residency and it is recoverable, but it is smaller than exp 284 measured. A worker circulating a token throu…
• 285.2 · The worker that never sleeps
  A sixth of the isolate boundary is scheduling, not transport. Against the same inline reference, exp 284's floor writer costs 5.117 us of boundary wi…
• 285.3 · The worker that never sleeps
  No library-side form of keeping a worker warm pays for itself, because the cheapest keep-alive Dart exposes costs more per turn than the residency it…
• 285.4 · The worker that never sleeps
  The release suite's Single Inserts (100 sequential) row charges resqlite for a caller-side wake that a real application does not pay. The lane awai…
• 285.5 · The worker that never sleeps
  The price of keeping a Dart isolate's event loop occupied, which is what any anti-parking proposal must pay. A token the isolate sends to its own rec…

What this changed

We believed the 1.8 us of claim 284.3 -- the identical executeWrite costing 7.612 us inline and 9.380 us on a just-woken worker -- was simply what a wake costs, with no statement about whether it is recoverable. Claim 285.1 makes it recoverable and smaller: a worker that never stops being runnable executes the same C call at 8.974 us against a parked worker's 9.606 us, so 0.63-0.88 us of it is thread residency the VM releases when an isolate goes idle. Read 284.3's 1.8 us as an upper bound measured on a single alternating pair, not as the residency cost of a continuously running lane.

We believed the isolate boundary claim 284.1 prices at 5.30 us was a transport cost. Claim 285.2 shows a sixth of it is scheduling: with neither side of the round trip parked, the same floor writer's boundary falls from 5.117 us to 4.225 us. This does not reopen the write path -- claim 285.3 closes every library-side form of collecting it -- but it does redirect where the remaining headroom is. A proposal that makes messages smaller or handlers leaner is working on the part that was already measured small; the part that is large is two thread wakes, and a package cannot address those.

New, and the finding a reader of the release suite most needs: claim 285.4 says the Single Inserts (100 sequential) row -- the one public lane where resqlite loses to a peer -- awaits each write and does nothing in between, so its caller parks and is woken 100 times. About a sixth of the boundary that row charges resqlite is the caller's own wake, which an application issuing the same writes from an isolate that has frames and futures pending does not pay. Anyone citing that row as the size of resqlite's write overhead, in benchmark/SCOPE.md or in a future experiment's Problem section, should cite it as a worst case rather than a typical one. The row is still fair; it is not representative.

We had no price for keeping a Dart event loop occupied, and exps 271 and 284 both reasoned about worker scheduling without one. Claim 285.5 supplies it: a self-sent port message turn costs 1.85 us on a busy isolate and 0.65 us on an idle one, a zero-duration Timer.run turn costs 6.50 us, and a microtask loop is disqualified outright because it starves the receive port the work arrives on. Any future proposal that would keep an isolate runnable is paying one of those three, and the first two are both larger than the residency they buy back.