Repository navigation
Do not silently select raw fallback when logical candidate costs are unavailable #721
Description
Problem
avg by (job) (data) exposes two separate failures in logical candidate selection. The semantic rewrite (sum by (job) (data) / count by (job) (data)) is generated, but the root silently resolves to raw KeepPreAsap / exact fallback.
Problem 1: absent costs become strategy-order selection
When alternatives have no usable comparable cost, selection falls back to candidate discovery order. The deployment registers SketchAlgorithmStrategy before SemanticEquivalentRewriteStrategy, so the former's Avg pass-through candidate wins over the latter's realizable sum / count rewrite. This is not an evidence-based choice.
Current effective behavior:
no comparable cost
→ preserve strategy registration/discovery order
→ pass-through appears before semantic rewrite
→ exact fallback is selected silently
Candidate resolution options
- Fail loudly (preferred for evidence-required planning): if a root has materially different legal alternatives and none can be compared, return a typed
cost_unavailableplanning error with the candidate identities and missing inputs. - Explicit deterministic policy: document and surface a deliberate fallback rule, such as preferring a realizable summary/rewrite over
KeepPreAsap. This is only valid if product policy accepts that choice without measured cost evidence. - Keep discovery order: not recommended; it makes registration order an undeclared optimizer policy.
Problem 2: ControlPlaneCostModel supplies no root-alternative cost
select_workload_impl calls Planner selection using ControlPlaneCostModel. That model currently does not implement a finite estimate_cost / candidate_cost for these logical root alternatives, and inherits the CostModel trait's NaN placeholder. The selection explain trace serializes this as:
"estimated_cost": null,
"estimated_cost_status": "not_reported_by_cost_model"The backend has some later-stage lifecycle/window evidence, but it does not currently produce a complete comparison between:
raw exact fallback
vs.
maintain/read Sum + maintain/read Count + final divide
Candidate resolution options
- Fail loudly (preferred until complete costs exist): have
ControlPlaneCostModel::candidate_costreturn unavailable for unpriced root alternatives, and make selection fail with a typed diagnostic rather than treatingNaNas a tie. - Implement complete-plan costing: provide finite, comparable costs for the raw baseline and every reachable rewritten candidate, including maintenance, readout, final exact operations, query/evaluation rate, and selected window framework. This is the long-term fix.
Acceptance criteria
- Candidate selection never silently treats non-finite/unavailable cost as an equal-cost tie.
- The trace identifies the missing cost inputs and the policy/error that resolved the group.
- A regression case for
avg by (job) (data)proves that unavailable costs produce the chosen typed error or an explicitly documented policy outcome. - If complete costs are supplied, the selected root is demonstrably the lower-cost complete plan.
Activity
milindsrivastava1997 commentedon Sep 14, 2026
Why complete workload quotes cannot currently choose the rewrite
The
avg by (job) (data)rewrite is not later overwritten by physical planning; it is discarded earlier, during logical root selection.Current sequence:
Therefore existing
workload_cost_evidencecannot make the rewrite win: it is consulted only after the rewrite no longer exists in the physical candidate inventory.The desired sequence is:
This also clarifies the two failure modes in this issue:
ControlPlaneCostModeldoes not supply a finite root-alternative cost for this rewrite, while complete snapshot quotes arrive too late to participate in root selection.An interim evidence-required policy could fail with a typed unavailable-cost error at step 2. The long-term solution is to move the decision to step 4, where complete plan costs can be compared.
milindsrivastava1997 commentedon Sep 14, 2026
can you please check this @zzylol ^