Repository navigation
Commit dedb5cf
fix(proto): decode HashJoinExec fetch with a checked u64 -> usize
This branch is stacked on the `fetch` fix and, after the rebase onto
main, carries its own copy of the `HashJoinExec` decode path. Apply the
same correction here so the defect does not survive on this branch and
reappear once #24165 merges.
Review feedback on #24165: `u64 as usize` silently truncates on a 32-bit
target, where `usize` is 32 bits. A fetch of `1 << 32` decodes to `0` --
not merely a wrong limit but the worst possible one, since "fetch 0
rows" turns the query into an empty result instead of erroring.
Use `usize::try_from` and surface an out-of-range value as an error.
Truncating and saturating both misrepresent the plan; an explicit decode
failure is the honest outcome, and it is only reachable on a 32-bit
target with an absurd fetch.
`plan_datafusion_err!` rather than `internal_datafusion_err!`: this
decode path reserves the internal-error macros for genuinely malformed
nodes (an unknown `PartitionMode` discriminant, a dynamic filter that
does not downcast), which really do indicate a bug. A well-formed but
unrepresentable `fetch` is not a DataFusion bug -- it is a plan that
cannot be expressed on this target -- and `plan_err!` is already this
file's idiom for invalid plan configuration.
The encode side (`fetch.map(|f| f as u64)`) needs no change: `usize` is
at most 64 bits on every supported target, so widening to `u64` is
lossless.
`roundtrip_hash_join_fetch` now also covers `u32::MAX as usize` and
`usize::MAX`, pinning that a large fetch round-trips exactly. Both are
representable on every target, so the test stays portable. The
truncation path itself is only reachable on a 32-bit target and is
therefore not covered on a 64-bit CI host.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>1 parent 36a7d74 commit dedb5cf
2 files changed
Lines changed: 30 additions & 7 deletions
File tree
- datafusion
- physical-plan/src/joins/hash_join
- proto/tests/cases
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1896 | 1896 | | |
1897 | 1897 | | |
1898 | 1898 | | |
1899 | | - | |
| 1899 | + | |
1900 | 1900 | | |
1901 | 1901 | | |
1902 | 1902 | | |
| |||
1978 | 1978 | | |
1979 | 1979 | | |
1980 | 1980 | | |
| 1981 | + | |
| 1982 | + | |
| 1983 | + | |
| 1984 | + | |
| 1985 | + | |
| 1986 | + | |
| 1987 | + | |
| 1988 | + | |
| 1989 | + | |
| 1990 | + | |
| 1991 | + | |
| 1992 | + | |
| 1993 | + | |
| 1994 | + | |
| 1995 | + | |
| 1996 | + | |
| 1997 | + | |
| 1998 | + | |
| 1999 | + | |
| 2000 | + | |
1981 | 2001 | | |
1982 | 2002 | | |
1983 | 2003 | | |
1984 | 2004 | | |
1985 | 2005 | | |
1986 | 2006 | | |
1987 | | - | |
1988 | | - | |
1989 | | - | |
1990 | | - | |
1991 | | - | |
| 2007 | + | |
1992 | 2008 | | |
1993 | 2009 | | |
1994 | 2010 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
563 | 563 | | |
564 | 564 | | |
565 | 565 | | |
566 | | - | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
567 | 574 | | |
568 | 575 | | |
569 | 576 | | |
| |||
0 commit comments