Skip to content

test: generate NULLs in equivalence fuzz tests - #26086

Open
Nagato-Yuzuru wants to merge 4 commits into
apache:mainfrom
Nagato-Yuzuru:fuzz-equivalence
Open

Nagato-Yuzuru wants to merge 4 commits into
apache:mainfrom
Nagato-Yuzuru:fuzz-equivalence

Conversation

@Nagato-Yuzuru

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Part of #24243

Rationale for this change

The equivalence fuzz tests never generated NULLs. So they could not catch NULL-ordering bugs in SortProperties propagation such as #11596 and #24208.

What changes are included in this PR?

  • Generated columns contain NULLs at configurable rates (NULL_PCTS = [0.0, 0.1, 0.5]), crossed with every seed; the schema is nullable only when the rate is non-zero.
  • create_random_schema picks random SortOptions for each ordering key.
  • Requirements with random SortOptions are checked only for soundness: if ordering_satisfy accepts an ordering, the data must be sorted by it. The converse is not checked, since one batch can be sorted by coincidence (e.g. NULLs of independent columns landing in the same rows). For the same reason, contains_overflowable_arithmetic is renamed to contains_conservative_ordering_op and also covers comparisons and AND/OR.

What is the testing strategy for this PR?

Test-only change. Verified locally that reverting the fixes from #24276 and #24206 makes test_ordering_satisfy_with_equivalence_complex_random fail, and that it passes on main:

git revert -n ceaef16778f3422312cc1e22ddf40e8f6acda56b 4042812a86f4a753c4b717ecbe12dbc928957254
cargo test -p datafusion --test fuzz --features extended_tests equivalence
#  test_ordering_satisfy_with_equivalence_complex_random fails

Are there any user-facing changes?

No

Refactor fuzz tests to replace `contains_overflowable_arithmetic` with `contains_conservative_ordering_op`, extending operator handling to include comparisons, logical operators, and arithmetic. Add new test cases for conservative operator logic verification.
@github-actions github-actions Bot added the core Core DataFusion crate label Oct 6, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.72%. Comparing base (31a4ca0) to head (4c5cdc6).
⚠️ Report is 214 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26086      +/-   ##
==========================================
+ Coverage   82.42%   82.72%   +0.29%     
==========================================
  Files        1139     1147       +8     
  Lines      435357   448179   +12822     
  Branches   435357   448179   +12822     
==========================================
+ Hits       358844   370740   +11896     
- Misses      54812    54902      +90     
- Partials    21701    22537     +836     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants