Repository navigation
Two worker slots stay free for jobs that make no model call, so a sync runs even while model work holds every other slot - #1349
Merged
Conversation
…c runs even while model work holds every other slot Signed-off-by: Quintin Botes <quintinbotes@outlook.com>
gedaiu
added a commit
to re-cinq/utopia
that referenced
this pull request
Oct 21, 2026
Sync upstream dev into re-dev (2026-10-21): deeplethe#1349, deeplethe#1280, deeplethe#1331, deeplethe#1326, deeplethe#1328
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1300: the reserved lane, the second of the two PRs agreed there. The first was #1324.
Why
#1324 orders the claim so that work which makes no chat call goes first. Ordering only decides who gets the next free slot, though. An extraction waiting for its model permit still holds its worker slot, and on a slow local model every slot is often held that way. A sync then waits for one of those extractions to finish, however far ahead of them it stands in the queue.
What changes
jobs::NO_CHAT_LANE = 2. Once allworker_concurrencyregular slots are taken,run_workermay still claim up to two more jobs, and only those withjob_priority(kind) = 0: the same list as Intake jobs are claimed before model work, so a sync no longer waits behind the extraction backlog #1324, so there is one list, not two. Each job returns its slot to the side it was claimed on (in_laneis counted apart fromrunning).worker_concurrencynow bounds model work, plus two slots reserved for intake jobs, as agreed in A sync or a new document waits behind the whole extraction backlog: jobs are claimed strictly by run_at #1300.claim_one(pool, no_chat_only). The lane's restriction is written into the SQL as a literal rather than a bound parameter. With a literal,EXPLAINshows it as an index condition onjobs_priority_claim_idx, so a lane claim stops at the end of the priority-0 rows. As a bound parameter it would be a row filter, scanning every queued extraction every 200 ms while the regular slots are full.en.tsandzh.ts) and the doc comment onDeploymentReq::worker_concurrencynow say it bounds model work, plus two slots reserved for intake jobs. Indocs/,worker_concurrencyonly appears in ADR 0006, which records the state at the time, so I've left it unchanged.Known limits
process_document, such as a big transcription, can still hold a lane slot for as long as it runs. As agreed in A sync or a new document waits behind the whole extraction backlog: jobs are claimed strictly by run_at #1300, that's acceptable for now.embed_ontologyandmemory_ingestcount as no-chat but wait on the embedding gate, as you noted in A sync or a new document waits behind the whole extraction backlog: jobs are claimed strictly by run_at #1300. Both can sit in the lane while they wait. That's still better than waiting behind chat work, and I've left it as it is.Test
tests/a_sync_runs_while_model_work_holds_every_slot.rsruns a realrun_worker, so it is a binary of its own, with a line intests/store/main.rs. The handler accepts only its own two marked jobs and fails anything else, ashuman_phrase_materialization_deliverydoes.worker_concurrencyat 1, anextract_documenttakes the only regular slot and never returns, standing in for an extraction waiting on the model.sync_sourcequeued after it must still run within ten seconds.NO_CHAT_LANE = 0the test fails:the sync waited behind the extraction: Err(Elapsed(())).How it was checked
On an M1 Max (macOS), at the PR head on top of
devf2e2a47:cargo fmt --all --check: passes.cargo clippy --workspace --all-targets -- -D warnings: no issues.cargo test --workspace --no-fail-fastwithUTOPIA_DATABASE_URLpointing at a database created and migrated for this branch alone: 1,646 passed, 0 failed, 6 ignored.pnpm buildandpnpm testinweb/: build passes, 307 tests pass.Before review
git commit -s)cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warningsandcargo test --workspacepassweb/:pnpm buildandpnpm testpasscrates/utopia-store/was tested withUTOPIA_DATABASE_URLset (those tests skip without it)web/src/i18n/en.tsandzh.ts