Conversation
`avg by (job) (data)` resolved to a raw `KeepPreAsap` pass-through, not because anything priced it lower, but because nothing priced it at all. Two legal alternatives reach that root — `SketchAlgorithmStrategy`'s `Avg` pass-through, which has no summary realization, and `SemanticEquivalentRewriteStrategy`'s realizable `sum / count` rewrite — and the cost model reports no comparable cost for either. Selection then preserved candidate discovery order, and the deployment registers the sketch strategy first, so the raw arm won. Strategy registration order was acting as undeclared optimizer policy. Selection now returns `SelectionError::CostUnavailable` for that group, carrying the target identity, every candidate's identity, strategy, replacement kind and provenance, so a reader can see which cost inputs the model owes rather than only that ranking failed. The selection trace records the same resolution under `unresolved_group`. The guard is deliberately narrow. It fires only when the discovery-order winner keeps the subtree pre-ASAP *and* a realizable alternative sits behind it unranked — the silent raw fallback the report is about. A group whose order-chosen candidate is already realizable still plans: `sum(sum_over_time(...))`, for instance, has the same unpriced mixed-kind shape but degrades to nothing. Detection matches on `SummaryExpr::KeepPreAsap` rather than on rationale text, so rewording upstream cannot silently disable it. Two process e2e workloads dropped `avg`, which no longer plans by design. Its behaviour is asserted at the selection layer instead, where the typed error is the observable outcome. The `average_overflow` fixture keeps `avg` under an Exact target, where the candidate set does not produce this shape. Complete-plan costing remains the long-term fix: with finite comparable costs these roots rank on evidence and the guard never fires. Closes #721. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #721.
What was happening
avg by (job) (data)resolved to a rawKeepPreAsappass-through — not because anything priced it lower, but because nothing priced it at all. Reproduced from the selection trace before any change:SketchAlgorithmStrategynull/not_reported_by_cost_modelSemanticEquivalentRewriteStrategynull/not_reported_by_cost_modelRank 0's own rationale says it: "
Avghas no summary realization and stays a logical pass-through." Rank 1 is the realizablesum / countrewrite. With no comparable cost, selection preserved candidate discovery order, and the deployment registers the sketch strategy first — so the raw arm won. Strategy registration order was acting as undeclared optimizer policy.The fix
Per the issue's preferred resolution, selection now fails loudly instead of resolving that group from discovery order:
The message names the target and both strategies, and each candidate carries its explain identity, so a reader can see which cost inputs the model owes rather than only that ranking failed. The selection trace records the same resolution under
unresolved_group(reason,policy,detail).The guard is deliberately narrow
My first cut fired on every group with two or more materially different unpriced alternatives — the issue's literal wording — and broke 23 existing tests. A survey showed why:
avg by (job) (data)sum(sum_over_time(...))sum(sum_over_time(...))has the same unpriced mixed-kind shape, but the order-chosen candidate is already realizable — nothing degrades to raw, so nothing is silently lost. The distinguishing signal is exactly what the issue title names: the discovery-order winner keeps the subtree pre-ASAP while a realizable alternative sits behind it unranked.So the guard requires all four of:
SummaryExpr::KeepPreAsap;Condition 3 matches on the post-ASAP IR, not on rationale prose, so rewording upstream cannot silently disable it.
Coverage change
Two process-e2e workloads dropped
avg, which no longer plans by design —current_series_process(avg(a),avg by (job) (a)) andissue_701_702_process(avg_over_time(...),avg(...)). They plan their whole query set as one workload, so an unplannable root takes the entire test down; keepingavgthere would have traded all their serving coverage for one assertion.avg's behaviour is asserted at the selection layer instead, where the typed error is the observable outcome. Theaverage_overflowfixture keepsavgunder anExacttarget — that candidate set does not produce this shape, and the test still passes.This is the intended consequence of failing loudly: those workloads legitimately cannot plan until the cost model prices these roots. Complete-plan costing remains the long-term fix, and with finite comparable costs the guard never fires.
Acceptance criteria
unresolved_groupplus per-candidate identity/strategy/kind/provenance.avg by (job) (data)—unpriced_alternatives_fail_instead_of_resolving_on_discovery_orderasserts the typed error, both strategy names, and that the group is genuinely mixed-kind (summary and rewrite), not two rankings of one shape.priced_roots_still_selectguards the other direction, including that a resolved selection carries nounresolved_group.Testing
cargo +1.98.0 fmt -- --check— clean.cargo +1.98.0 clippy --workspace --all-targets -- -D warnings— clean.cargo +1.98.0 test --workspace -- --test-threads=1— exit 0, zero failures across 27 test binaries.🤖 Generated with Claude Code