Skip to content

feat(lower): support *_over_time over a sub-query (#27) - #42

Merged
zzylol merged 1 commit into
mainfrom
feat/over-time-of-subquery
Jul 2, 2026
Merged

feat(lower): support *_over_time over a sub-query (#27)#42
zzylol merged 1 commit into
mainfrom
feat/over-time-of-subquery

Conversation

@zzylol

@zzylol zzylol commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What & why

Follow-up to #27. The *_over_time / quantile_over_time-over-sub-query family — the canonical

max_over_time(rate(m[5m])[1h:])

— was rejected:

unsupported feature: expected a range-vector (matrix) argument, got Discriminant(4)

extract_matrix only accepts a (parenthesised) matrix selector, not the PromQLSubquery a sub-query argument lowers to. This was pinned as over_time_of_subquery_is_rejected__GAP (the last un-issued item on #27's checklist after topk-bare-by and SQL derived tables landed).

How

crates/lower/src/promql.rs — a new walk_call intercepts a *_over_time/quantile_over_time call whose argument is a sub-query, lowers the sub-query recursively (PromQLSubquery), and reduces it per series. Non-sub-query calls fall through to the existing flat lower_inner_call template unchanged:

max_over_time(rate(m[5m])[1h:])
→ Aggregate{Max} → Subquery{1h} → Aggregate{Rate} → TimeRange{5m} → Scan

over_time_reducer reuses inner_func (no AggFunc drift); quantile_over_time's φ is read from arg 0, the sub-query from arg 1.

crates/core/src/intent_algebra/query_expr.rs — an empty-keys single aggregate over a Subquery is now recognised as a label-preserving per-series range reduction, mirroring the existing TimeRange marker.

Why the per-series marker is sound

An empty-keys aggregate over a Subquery would otherwise be treated as a cross-series reduction and collapse every label. That's safe to change because a genuine cross-series aggregation operator over a range vector — sum(rate(m[5m])[1h:]) — is a PromQL type error the parser already rejects (expected type vector in aggregation expression, got matrix). So an Aggregate directly over a Subquery only ever arises from this per-series *_over_time shape.

Confirmed by sum by (job) (max_over_time(rate(m[5m])[1h:])): the inner Max preserves the job label so the outer sum by (job) resolves and groups on it.

Tests

Flips over_time_of_subquery_is_rejected__GAP into three passing conformance tests:

  • over_time_of_subquery_reduces_per_series — per-series Max over a Subquery, no group keys, inner Rate present.
  • quantile_over_time_of_subquery_carries_phi — φ = 0.9 threaded from arg 0, sub-query from arg 1.
  • aggregation_over_over_time_of_subquery_keeps_labels — outer sum by (job) groups on the label the inner reduction preserved.

Regression-checked: plain max_over_time(m[5m]), heavy-hitter topk(k, count_over_time(...)), and parser-rejected sum(subquery) unchanged. Full workspace suite green; cargo clippy --all-targets clean.

Notes

Branches off latest main (includes #31/#32). Advances #27 — the remaining checklist items there are now the SQL scalar/IN/EXISTS subqueries and the scalar/negation operands (tracked by #35/#36).

🤖 Generated with Claude Code

`max_over_time(rate(m[5m])[1h:])` and the rest of the
`*_over_time`/`quantile_over_time`-of-sub-query family were rejected —
`extract_matrix` only accepts a (parenthesised) matrix selector, not the
`PromQLSubquery` a sub-query argument produces.

`walk_call` now intercepts a `*_over_time` call whose argument is a
sub-query, lowers the sub-query recursively, and reduces it per series:

  max_over_time(rate(m[5m])[1h:])
  -> Aggregate{Max} -> Subquery{1h} -> Aggregate{Rate} -> TimeRange{5m} -> Scan

Non-sub-query calls fall through to the existing flat template unchanged,
so plain `max_over_time(m[w])` and heavy-hitter
`topk(k, count_over_time(m[w]))` are untouched.

Per-series correctness: an empty-keys aggregate over a `Subquery` is now
recognised as a label-preserving per-series range reduction, mirroring the
existing `TimeRange` marker. This is sound because a genuine cross-series
aggregation operator over a range vector (`sum(rate(m[5m])[1h:])`) is a
PromQL type error the parser already rejects — so an `Aggregate` directly
over a `Subquery` only ever arises from this per-series `*_over_time` shape.
Verified by `sum by (job) (max_over_time(rate(m[5m])[1h:]))`: the inner
`Max` preserves `job` for the outer `sum by (job)` to group on.

Flips the `over_time_of_subquery_is_rejected__GAP` conformance test into
three passing tests (per-series reduction, `quantile_over_time` phi, outer
aggregation label preservation). Full workspace suite green; clippy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@zzylol
zzylol merged commit 2628423 into main Jul 2, 2026
1 check passed
@zzylol
zzylol deleted the feat/over-time-of-subquery branch July 2, 2026 14:04
zzylol pushed a commit that referenced this pull request Jul 2, 2026
…cs (#44)

Pin how the new counter-derivative intents compose with the rest of the
lowering — they are per-series/label-preserving, so they reuse the general
nesting path with no special-casing:

- outer cross-series stat over a derivative nests two levels and threads
  the outer group key + any scalar param (`avg by (dc)(predict_linear(...))`)
- `topk(k, deriv(...))` is generic Sort+Limit, not a heavy-hitter TopK
- as a binary-op operand and under `sum(rate(...) + changes(...))`
- counter-derivative over a sub-query stays cleanly rejected (a #42 follow-up)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Jul 2, 2026
* feat(lower): counter-derivative range functions (#44)

`changes`, `delta`, `idelta`, `deriv`, `resets`, `predict_linear`, and
`double_exponential_smoothing` (a.k.a. `holt_winters`) parsed but were
rejected (`UnsupportedFunction`). They now lower to distinct per-series
intents — the second-largest un-tracked lowering gap (108 corpus
rejections, 5.6%).

Each is a per-series, label-preserving reduction of one series' range
window to one value, riding on the enclosing `TimeRange`:

  changes(v[w]) -> Aggregate{Changes, by:[]} -> TimeRange{w} -> Scan

Deliberately NOT aliased to rate/increase/count: `changes` (value-change
count) and `resets` (counter-reset count) are distinct from a sample
count; `delta`/`idelta`/`deriv` are gauge derivatives. `predict_linear`
carries its horizon (`seconds`); `double_exponential_smoothing` carries
both smoothing factors; `holt_winters` maps to the same intent.

Pipeline:
- L3 `AggIntent`: + Changes/Delta/IDelta/Deriv/Resets/PredictLinear/
  DoubleExpSmoothing; requires()=TimeSeries, is_per_series()=true,
  output_column() named after the function (float64).
- L2 `AggFunc`: mirror variants (window rides on the L2 Window node, like
  *_over_time, so — unlike Rate/Increase — they don't carry it).
- Converter `agg_func_to_intent`: map each, threading scalar params.
- PromQL front end: `InnerFunc` variants + `lower_inner_call` arms
  (`predict_linear` reads arg1; `double_exponential_smoothing`/
  `holt_winters` read args 1-2).

Tests:
- promql_conformance.rs section M — distinct intents, predict_linear
  horizon, double_exp/holt_winters alias equivalence, label preservation
  under an outer `sum by`.
- awesome_prometheus_alerts.rs — 3 __GAP tests flipped to positive
  (function body lowers; the full `... > N` alert still needs #35).
- promql_equivalence.rs — changes/resets upgraded from rejected to
  pairwise-distinct vs count_over_time.
- promql_conformance.rs — dropped the 5 now-supported entries from
  unsupported_functions_are_rejected.

Full workspace suite green; clippy --all-targets clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(lower): composition of counter-derivatives with outer/nested funcs (#44)

Pin how the new counter-derivative intents compose with the rest of the
lowering — they are per-series/label-preserving, so they reuse the general
nesting path with no special-casing:

- outer cross-series stat over a derivative nests two levels and threads
  the outer group key + any scalar param (`avg by (dc)(predict_linear(...))`)
- `topk(k, deriv(...))` is generic Sort+Limit, not a heavy-hitter TopK
- as a binary-op operand and under `sum(rate(...) + changes(...))`
- counter-derivative over a sub-query stays cleanly rejected (a #42 follow-up)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: zz_y <zz_y@node0.zz-y-308294.softmeasure-pg0.clemson.cloudlab.us>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Jul 2, 2026
…76)

Generalizes the `*_over_time`-over-sub-query support (#42) to the whole
range-vector family: `rate`/`irate`/`increase` and the counter-derivatives
(`changes`/`delta`/`idelta`/`deriv`/`resets`/`predict_linear`/
`double_exponential_smoothing`). `f(<inst>[range:res])` now lowers to a
per-series `Aggregate{[f]}` directly over the `PromQLSubquery`.

Front end: `walk_call` routes any range function whose matrix argument is a
sub-query through a unified `range_fn_over_subquery` (rate/increase read the
window from the sub-query's own range; predict_linear/double_exp thread
their scalar params; quantile_over_time reads φ from arg 0).

Converter: a `PromQLSubquery` input is the range context, so (1) no
`TimeRange` is synthesized over it — `rate(sub)` would otherwise get a
spurious TimeRange from its carried window — and (2) the #71 counter-
derivative range-less guard correctly does not fire when the input is a
sub-query.

Flips `counter_derivative_over_a_subquery_is_rejected__GAP` into passing
conformance tests (per-series intent directly over the Subquery, no
TimeRange; scalar params carried). Normal matrix forms (`rate(m[5m])`,
`changes(m[15m])`) unchanged. Full workspace suite green; clippy clean.

Co-authored-by: zz_y <zz_y@node0.zz-y-308294.softmeasure-pg0.clemson.cloudlab.us>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant