Repository navigation
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #25364 +/- ##
==========================================
+ Coverage 82.59% 82.72% +0.12%
==========================================
Files 1145 1147 +2
Lines 444096 448217 +4121
Branches 444096 448217 +4121
==========================================
+ Hits 366806 370781 +3975
+ Misses 55021 54898 -123
- Partials 22269 22538 +269 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
7666e87 to
f12c31d
Compare
Preserve known parameter fields before applying row-count defaults and keep incomplete PREPARE signatures deferred. Fixes apache#15978 Signed-off-by: 1fanwang <1fannnw@gmail.com>
f12c31d to
3104b5b
Compare
Signed-off-by: 1fanwang <1fannnw@gmail.com>
jayzhan211
left a comment
There was a problem hiding this comment.
Thanks @1fanwang , here are some suggestions
coerce_limit_expr cast-wrapped untyped LIMIT/OFFSET placeholders, hiding them from get_parameter_fields's row-count-default pass, which only matches a direct Limit-node placeholder, so optimized get_parameter_types() returned None instead of Some(Int64). Leave bare untyped placeholders unwrapped. Add a prepare.slt case mixing a typed filter with an untyped LIMIT. Signed-off-by: 1fanwang <1fannnw@gmail.com>
jayzhan211
left a comment
There was a problem hiding this comment.
Thanks @1fanwang , 1 suggestion
Signed-off-by: 1fanwang <1fannnw@gmail.com>
jayzhan211
left a comment
There was a problem hiding this comment.
The only request left: could we minimize the tests? Most of the 371 lines fit existing harnesses:
- metadata, type-conflict and field-metadata cases →
ParameterTest/ParameterTestWithMetadataindatafusion/sql/tests/cases/params.rs(it checks types, binding and thePreparefields) - EXECUTE result cases →
prepare.slt - the analyzer→bind→optimize guard →
datafusion/optimizer/tests/optimizer_integration.rs
Only the named-parameter (DataFrame API) case needs to stay in core_integration. While moving them, please drop the leftover println! calls
Record bare row-count parameter fields before analyzer coercion instead of inferring their inputs through arbitrary Int64 casts. Preserve types inferred elsewhere and string binding through explicit BIGINT casts. Signed-off-by: 1fanwang <1fannnw@gmail.com>
| } | ||
|
|
||
| #[tokio::test] | ||
| async fn test_limit_parameter_binding_with_inferred_type() -> Result<()> { |
There was a problem hiding this comment.
There is still quite a bit of parameter-test setup duplicated here. Could you move the planning and binding assertions into the existing parameter harnesses where they fit, while keeping the analyzer/optimizer and execution-specific cases in coverage that actually exercises those phases?
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Which issue does this PR close?
LIMITclause #15978.Rationale for this change
Clients preparing queries need usable parameter types. Bare LIMIT/OFFSET parameters should default to Int64, but an explicit CAST($1 AS BIGINT) must leave its input unknown because the cast also accepts strings. Reporting Int64 incorrectly tells clients they must supply an integer.
Previously, optimized plans lost the bare parameter's default. Looking through every Int64 cast restored it but incorrectly typed explicit cast inputs.
What changes are included in this PR?
The analyzer records bare LIMIT/OFFSET parameter fields before coercion, preserving a type inferred elsewhere. Lookup does not unwrap user-written casts. This keeps explicit cast inputs unknown and allows Int32 binding followed by Int64 LIMIT coercion.
Coverage uses the existing parameter harnesses, optimizer integration, prepare.slt, and named/metadata DataFrame execution tests.
What is the testing strategy for this PR?
Testing Done
macOS arm64, Rust 1.98.1, with the repository's test-data submodules initialized.
Before the explicit-cast fix, the BIGINT cases failed on the production code from b66d80b. The regression then lived in core integration:
cargo test --profile ci -p datafusion --test core_integration test_limit_offset_parameters_leave_cast_inputs_unresolved -- --nocaptureAfter the fix, the real CLI accepts string inputs in LIMIT, OFFSET and nested explicit casts, and paging returns the requested row:
cargo run --profile ci -p datafusion-cli -- --quiet -c "PREPARE page AS SELECT value FROM (VALUES (10), (20), (30)) AS t(value) ORDER BY value LIMIT \$1 OFFSET \$2; EXECUTE page(1, 1); DEALLOCATE page; PREPARE cast_page AS SELECT \$1 AS value LIMIT CAST(\$1 AS BIGINT); EXECUTE cast_page('1'); DEALLOCATE cast_page; PREPARE cast_offset AS SELECT \$1 AS value FROM (VALUES (1), (2)) AS t(v) OFFSET CAST(\$1 AS BIGINT); EXECUTE cast_offset('1'); DEALLOCATE cast_offset; PREPARE nested_cast AS SELECT \$1 AS value FROM (SELECT 1 LIMIT CAST(\$1 AS BIGINT)) AS t; EXECUTE nested_cast('1'); DEALLOCATE nested_cast;"Are there any user-facing changes?
Bare LIMIT/OFFSET parameters default to Int64 before and after optimization. Explicit casts leave their input parameter types unresolved.