Skip to content

fix: handle nested RANGE CURRENT ROW peers - #26067

Open
sunchao wants to merge 2 commits into
apache:mainfrom
sunchao:fix/nested-range-peer-boundaries
Open

sunchao wants to merge 2 commits into
apache:mainfrom
sunchao:fix/nested-range-peer-boundaries

Conversation

@sunchao

@sunchao sunchao commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Which issue does this PR close?

Related to #24937. This fixes the RANGE-window case; general MIN/MAX scalar comparisons remain separate.

Rationale for this change

A RANGE window ordered by a list containing lists or structs can plan successfully and then fail with Uncomparable values. For example:

SELECT count(*) OVER (ORDER BY l)
FROM (VALUES ([[1]]), ([[2]])) t(l);

CURRENT ROW boundaries only need to find equal ordering keys in the sorted input.

What changes are included in this PR?

Use Arrow's comparator to find nested CURRENT ROW peer boundaries. Reuse the comparator while the ordering arrays are unchanged and release it when buffered rows are pruned. Preserve WindowFrameStateRange's existing unwind-safety traits.

SQL type admission, scalar-key handling, and finite-offset calculations are unchanged.

What is the testing strategy for this PR?

Update the two existing SQL regressions from expected errors to successful counts. Physical-operator tests cover nested lists/structs, nulls, tied ordering keys across batches, frame boundaries, and completed-peer emission before EOF in sorted mode. A small state test covers comparator reuse, pruning, and the public unwind-safety trait guarantees.

Local validation on this patch:

  • Full extended workspace suite: 12,421 Rust tests passed, eight ignored, and all 527 SQL-logic files completed.
  • All-target/all-feature Clippy with warnings denied passed. The downstream catch_unwind compatibility reproducer also compiles for both borrowed and owned window state.
  • Formatting, license/typo checks, generated documentation checks, local link checks, and the security audit passed. API and website documentation builds also passed with warnings treated as errors.
  • Full lint could not complete locally: the package mirror does not offer cargo 0.100.0, pinned by the existing dev/depcheck tool. This blocks the circular-dependency check and that tool's metadata inspection by cargo-machete; cargo-machete otherwise reported no unused dependencies. This patch changes no dependency manifests or lockfiles.
  • The physical regression and both SQL regressions fail on the unpatched target base with the original comparison error.
  • All six existing aggregate-window benchmark smoke cases pass on both base and candidate. No throughput measurements are claimed.

Are there any user-facing changes?

Already-admitted SQL RANGE windows over nested list keys return results instead of an internal comparison error. The same boundary fix applies to windows constructed through the physical API. No new public API or SQL-admitted types are introduced.

@github-actions github-actions Bot added logical-expr Logical plan and expressions optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) auto detected api change Auto detected API change labels Oct 6, 2026
@codecov-commenter

codecov-commenter commented Oct 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.61905% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.72%. Comparing base (c0e872f) to head (f0d7123).
⚠️ Report is 20 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/expr/src/window_state.rs 87.25% 6 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26067      +/-   ##
==========================================
+ Coverage   82.69%   82.72%   +0.02%     
==========================================
  Files        1147     1147              
  Lines      447348   449092    +1744     
  Branches   447348   449092    +1744     
==========================================
+ Hits       369936   371509    +1573     
+ Misses      54999    54932      -67     
- Partials    22413    22651     +238     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot removed the auto detected api change Auto detected API change label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate logical-expr Logical plan and expressions optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants