feat(lower): support *_over_time over a sub-query (#27) - #42
Merged
Conversation
`max_over_time(rate(m[5m])[1h:])` and the rest of the
`*_over_time`/`quantile_over_time`-of-sub-query family were rejected —
`extract_matrix` only accepts a (parenthesised) matrix selector, not the
`PromQLSubquery` a sub-query argument produces.
`walk_call` now intercepts a `*_over_time` call whose argument is a
sub-query, lowers the sub-query recursively, and reduces it per series:
max_over_time(rate(m[5m])[1h:])
-> Aggregate{Max} -> Subquery{1h} -> Aggregate{Rate} -> TimeRange{5m} -> Scan
Non-sub-query calls fall through to the existing flat template unchanged,
so plain `max_over_time(m[w])` and heavy-hitter
`topk(k, count_over_time(m[w]))` are untouched.
Per-series correctness: an empty-keys aggregate over a `Subquery` is now
recognised as a label-preserving per-series range reduction, mirroring the
existing `TimeRange` marker. This is sound because a genuine cross-series
aggregation operator over a range vector (`sum(rate(m[5m])[1h:])`) is a
PromQL type error the parser already rejects — so an `Aggregate` directly
over a `Subquery` only ever arises from this per-series `*_over_time` shape.
Verified by `sum by (job) (max_over_time(rate(m[5m])[1h:]))`: the inner
`Max` preserves `job` for the outer `sum by (job)` to group on.
Flips the `over_time_of_subquery_is_rejected__GAP` conformance test into
three passing tests (per-series reduction, `quantile_over_time` phi, outer
aggregation label preservation). Full workspace suite green; clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
9 tasks
zzylol
pushed a commit
that referenced
this pull request
Jul 2, 2026
…cs (#44) Pin how the new counter-derivative intents compose with the rest of the lowering — they are per-series/label-preserving, so they reuse the general nesting path with no special-casing: - outer cross-series stat over a derivative nests two levels and threads the outer group key + any scalar param (`avg by (dc)(predict_linear(...))`) - `topk(k, deriv(...))` is generic Sort+Limit, not a heavy-hitter TopK - as a binary-op operand and under `sum(rate(...) + changes(...))` - counter-derivative over a sub-query stays cleanly rejected (a #42 follow-up) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Jul 2, 2026
* feat(lower): counter-derivative range functions (#44) `changes`, `delta`, `idelta`, `deriv`, `resets`, `predict_linear`, and `double_exponential_smoothing` (a.k.a. `holt_winters`) parsed but were rejected (`UnsupportedFunction`). They now lower to distinct per-series intents — the second-largest un-tracked lowering gap (108 corpus rejections, 5.6%). Each is a per-series, label-preserving reduction of one series' range window to one value, riding on the enclosing `TimeRange`: changes(v[w]) -> Aggregate{Changes, by:[]} -> TimeRange{w} -> Scan Deliberately NOT aliased to rate/increase/count: `changes` (value-change count) and `resets` (counter-reset count) are distinct from a sample count; `delta`/`idelta`/`deriv` are gauge derivatives. `predict_linear` carries its horizon (`seconds`); `double_exponential_smoothing` carries both smoothing factors; `holt_winters` maps to the same intent. Pipeline: - L3 `AggIntent`: + Changes/Delta/IDelta/Deriv/Resets/PredictLinear/ DoubleExpSmoothing; requires()=TimeSeries, is_per_series()=true, output_column() named after the function (float64). - L2 `AggFunc`: mirror variants (window rides on the L2 Window node, like *_over_time, so — unlike Rate/Increase — they don't carry it). - Converter `agg_func_to_intent`: map each, threading scalar params. - PromQL front end: `InnerFunc` variants + `lower_inner_call` arms (`predict_linear` reads arg1; `double_exponential_smoothing`/ `holt_winters` read args 1-2). Tests: - promql_conformance.rs section M — distinct intents, predict_linear horizon, double_exp/holt_winters alias equivalence, label preservation under an outer `sum by`. - awesome_prometheus_alerts.rs — 3 __GAP tests flipped to positive (function body lowers; the full `... > N` alert still needs #35). - promql_equivalence.rs — changes/resets upgraded from rejected to pairwise-distinct vs count_over_time. - promql_conformance.rs — dropped the 5 now-supported entries from unsupported_functions_are_rejected. Full workspace suite green; clippy --all-targets clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(lower): composition of counter-derivatives with outer/nested funcs (#44) Pin how the new counter-derivative intents compose with the rest of the lowering — they are per-series/label-preserving, so they reuse the general nesting path with no special-casing: - outer cross-series stat over a derivative nests two levels and threads the outer group key + any scalar param (`avg by (dc)(predict_linear(...))`) - `topk(k, deriv(...))` is generic Sort+Limit, not a heavy-hitter TopK - as a binary-op operand and under `sum(rate(...) + changes(...))` - counter-derivative over a sub-query stays cleanly rejected (a #42 follow-up) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: zz_y <zz_y@node0.zz-y-308294.softmeasure-pg0.clemson.cloudlab.us> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 2, 2026
zzylol
added a commit
that referenced
this pull request
Jul 2, 2026
…76) Generalizes the `*_over_time`-over-sub-query support (#42) to the whole range-vector family: `rate`/`irate`/`increase` and the counter-derivatives (`changes`/`delta`/`idelta`/`deriv`/`resets`/`predict_linear`/ `double_exponential_smoothing`). `f(<inst>[range:res])` now lowers to a per-series `Aggregate{[f]}` directly over the `PromQLSubquery`. Front end: `walk_call` routes any range function whose matrix argument is a sub-query through a unified `range_fn_over_subquery` (rate/increase read the window from the sub-query's own range; predict_linear/double_exp thread their scalar params; quantile_over_time reads φ from arg 0). Converter: a `PromQLSubquery` input is the range context, so (1) no `TimeRange` is synthesized over it — `rate(sub)` would otherwise get a spurious TimeRange from its carried window — and (2) the #71 counter- derivative range-less guard correctly does not fire when the input is a sub-query. Flips `counter_derivative_over_a_subquery_is_rejected__GAP` into passing conformance tests (per-series intent directly over the Subquery, no TimeRange; scalar params carried). Normal matrix forms (`rate(m[5m])`, `changes(m[15m])`) unchanged. Full workspace suite green; clippy clean. Co-authored-by: zz_y <zz_y@node0.zz-y-308294.softmeasure-pg0.clemson.cloudlab.us> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Follow-up to #27. The
*_over_time/quantile_over_time-over-sub-query family — the canonical— was rejected:
extract_matrixonly accepts a (parenthesised) matrix selector, not thePromQLSubquerya sub-query argument lowers to. This was pinned asover_time_of_subquery_is_rejected__GAP(the last un-issued item on #27's checklist after topk-bare-by and SQL derived tables landed).How
crates/lower/src/promql.rs— a newwalk_callintercepts a*_over_time/quantile_over_timecall whose argument is a sub-query, lowers the sub-query recursively (PromQLSubquery), and reduces it per series. Non-sub-query calls fall through to the existing flatlower_inner_calltemplate unchanged:over_time_reducerreusesinner_func(noAggFuncdrift);quantile_over_time's φ is read from arg 0, the sub-query from arg 1.crates/core/src/intent_algebra/query_expr.rs— an empty-keys single aggregate over aSubqueryis now recognised as a label-preserving per-series range reduction, mirroring the existingTimeRangemarker.Why the per-series marker is sound
An empty-keys aggregate over a
Subquerywould otherwise be treated as a cross-series reduction and collapse every label. That's safe to change because a genuine cross-series aggregation operator over a range vector —sum(rate(m[5m])[1h:])— is a PromQL type error the parser already rejects (expected type vector in aggregation expression, got matrix). So anAggregatedirectly over aSubqueryonly ever arises from this per-series*_over_timeshape.Confirmed by
sum by (job) (max_over_time(rate(m[5m])[1h:])): the innerMaxpreserves thejoblabel so the outersum by (job)resolves and groups on it.Tests
Flips
over_time_of_subquery_is_rejected__GAPinto three passing conformance tests:over_time_of_subquery_reduces_per_series— per-seriesMaxover aSubquery, no group keys, innerRatepresent.quantile_over_time_of_subquery_carries_phi— φ = 0.9 threaded from arg 0, sub-query from arg 1.aggregation_over_over_time_of_subquery_keeps_labels— outersum by (job)groups on the label the inner reduction preserved.Regression-checked: plain
max_over_time(m[5m]), heavy-hittertopk(k, count_over_time(...)), and parser-rejectedsum(subquery)unchanged. Full workspace suite green;cargo clippy --all-targetsclean.Notes
Branches off latest
main(includes #31/#32). Advances #27 — the remaining checklist items there are now the SQL scalar/IN/EXISTSsubqueries and the scalar/negation operands (tracked by #35/#36).🤖 Generated with Claude Code