Repository navigation
Schedule lazy scan preparation on the CPU executor - #97
Closed
phillipleblanc wants to merge 1 commit into
Closed
phillipleblanc wants to merge 1 commit into
phillipleblanc wants to merge 1 commit into
Conversation
Signed-off-by: Phillip LeBlanc <879445+phillipleblanc@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WHAT
Schedule lazy scan preparation through the CPU executor while preserving lazy, bounded split admission. This remains a draft experiment: CPU improves, but indexed 64-client p99 rises 65 → 75 ms, so the no-latency-regression gate is unmet.
WHY
Expression optimization and split preparation consume CPU and compete with file I/O when placed on the blocking pool.
HOW
Use the session CPU scheduling API and cover the first-poll executor choice with a test that fails before the change. All 304 layout/file nextest tests, scoped pinned-toolchain Clippy and formatting pass; built Spice from the exact fork and validated real fixture results/full scans plus paired 64-client and 3,000/s offered profiles.
Indexed CPU falls 4.569 → 4.377 ms/lookup; corrected samples show preparation leaves the blocking executor and pool-lock share falls 9.77% → 8.05%. Indexed p99.9 rises 102 → 123.7 ms. Do not merge until the tail-latency gate is resolved.
Detailed local evidence retains exact source/binary digests, raw profiles, commands, full results and attribution limits; artifacts are local, not uploaded. Runtime validation uses spiceai/spiceai#14149 and instrumentation spiceai/spiceai#14174; the fork diff is independent of #96. No lab signoff was run under the explicit instruction.