fix: Keep requested keys alive in the naive scheduler - #240
Merged
Conversation
NaiveScheduler frees an intermediate result as soon as every provider consuming it has run. Requested keys were not exempt from this, so a key that is both requested and consumed by another task was deleted before it could be returned, raising KeyError. This affects any compute for multiple targets where one target depends on another, for example `pipeline.compute((A, B))` with a provider `A -> B`. The dask scheduler is unaffected. Excluding the requested keys from the discard step keeps them alive until they are returned, while everything else is still discarded as early as before.
SimonHeybrock
force-pushed
the
naive-scheduler-keep-requested-keys
branch
from
August 31, 2026 06:30
0fc3dcc to
e1b3239
Compare
jl-wynen
reviewed
Sep 2, 2026
Comment on lines
+88
to
+97
| requested: Container[Hashable], | ||
| ) -> None: | ||
| """Discard results that no remaining provider needs. | ||
|
|
||
| Requested keys are kept: they are returned to the caller, so their consumer | ||
| count reaching zero does not mean they are no longer needed. | ||
| """ | ||
| for arg in provider.arg_spec.keys(): | ||
| counts[arg] -= 1 | ||
| if counts[arg] == 0: | ||
| if counts[arg] == 0 and arg not in requested: |
Member
There was a problem hiding this comment.
Suggested change
| requested: Container[Hashable], | |
| ) -> None: | |
| """Discard results that no remaining provider needs. | |
| Requested keys are kept: they are returned to the caller, so their consumer | |
| count reaching zero does not mean they are no longer needed. | |
| """ | |
| for arg in provider.arg_spec.keys(): | |
| counts[arg] -= 1 | |
| if counts[arg] == 0: | |
| if counts[arg] == 0 and arg not in requested: | |
| requested: set[Hashable], | |
| ) -> None: | |
| """Discard results that no remaining provider needs. | |
| Requested keys are kept: they are returned to the caller, so their consumer | |
| count reaching zero does not mean they are no longer needed. | |
| """ | |
| for arg in provider.arg_spec.keys() - requested: | |
| counts[arg] -= 1 | |
| if counts[arg] == 0: |
This is an internal function, so it doesn't need to have a general argument. This way, you avoid repeated lookups.
Member
Author
There was a problem hiding this comment.
Good point on the annotation, changed to set[Hashable].
I kept the loop as it was, though: ArgSpec.keys() is a generator, so keys() - requested raises TypeError -- making it work would need a set(...) construction per call. And the lookup isn't actually repeated: arg not in requested sits behind counts[arg] == 0 and short-circuits, so it runs at most once per provider, for the single argument whose count reaches zero.
_consume_arguments is internal and always called with a set.
jl-wynen
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NaiveSchedulerfrees an intermediate result as soon as every provider consuming it has run. The consumer count is built from the graph's providers only, so a key that is both requested and consumed by another task reaches zero and is deleted before it can be returned:Any compute for several targets where one target depends on another is affected; the dask scheduler is not. The regression came in with a5c9a08 ("Discard data early in naive scheduler") and is in 26.8.0, so 25.11.1 and earlier are fine.
Counting the requested keys as consumers keeps them alive until they are returned, and everything else is still discarded as early as before.
This was found from downstream fallout rather than from reading the code: ESSlivedata defaults its services to the naive scheduler, and there both detector-view construction and monitor workflow cycles fail with a
KeyErroron a live key.ess.reduce.streaming.StreamProcessorhits the pattern routinely, computing several targets at once where one feeds another. With this fix those workflows run again under the naive scheduler.Test plan: the new test runs on all three scheduler fixtures and fails only on
[naive]without the fix.