Repository navigation
test: cover CrossJoinExec row metrics across batched inputs - #25530
Conversation
Add a Rust test that asserts build_input_rows, input_rows, and output_rows stay stable when CrossJoinExec inputs are split into multiple batches. Related to apache#25333.
e12556c to
7c49814
Compare
|
@CuteChuanChuan I picked up the CrossJoinExec portion of #25333 in this PR. Since you outlined the join-metrics test approach, could you take a look at whether these batch-size cases and metric assertions cover what you had in mind? Thanks! |
|
Thank you for opening this pull request! Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch). Details |
kosiew
left a comment
There was a problem hiding this comment.
Thanks for adding this regression coverage. The new tests verify the cross-join row metrics across different input batch layouts and address the issue requirements. I have one optional suggestion to make the multi-batch setup even more explicit.
| assert_eq!(num_rows, 6); | ||
| assert_join_metrics!(metrics, 6); | ||
| assert_eq!(metric_count(&metrics, "build_input_rows"), 3); | ||
| assert_eq!(metric_count(&metrics, "input_rows"), 2); |
There was a problem hiding this comment.
Optional: could we also assert build_input_batches and input_batches, expecting (2, 1) for batch size 2 and (3, 2) for batch size 1? That would ensure future test setup changes do not accidentally remove the multi-batch coverage while keeping the same row totals.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #25530 +/- ##
=======================================
Coverage 82.73% 82.73%
=======================================
Files 1147 1147
Lines 449509 449568 +59
Branches 449509 449568 +59
=======================================
+ Hits 371893 371948 +55
- Misses 54944 54946 +2
- Partials 22672 22674 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Thanks @kosiew for the review and suggestion! I’ve added assertions for both batch metrics to ensure the test continues to cover the intended multi-batch scenarios. |
kosiew
left a comment
There was a problem hiding this comment.
Thanks for the follow-up. The additional assertions verify that build and probe row counts remain correct across different input batch layouts, while the batch counters reflect the actual number of batches. The changes are limited to tests and address the regression coverage gap.
LGTM!
|
🚀 |
Which issue does this PR close?
Part of #25333 (
CrossJoinExectest coverage only).Rationale for this change
Join metrics can be incorrect without affecting query results. Existing
CrossJoinExectests did not assertbuild_input_rowsorinput_rowsacross different input batch layouts.What changes are included in this PR?
Adds Rust test coverage that:
build_input_rows,input_rows, andoutput_rowsfor the existing single-batch case.This is a test-only change.
What is the testing strategy for this PR?
cargo fmt --check cargo test -p datafusion-physical-plan --lib joins::cross_join cargo clippy -p datafusion-physical-plan --all-targets --all-features -- -D warningsAll checks pass locally.
Are there any user-facing changes?
No.