You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Perfect hash join panics with overflow on full-range signed integer keys #20995
Comet's CometFuzzTestSuite"join" test uses a mix of extreme and random values to exercise operators, which found this bug while testing DF 53.0.0-rc2 (apache/datafusion-comet#3629).
try_build_perfect_hash_map panics with attempt to add with overflow when a signed integer join key spans the full range of its type (e.g. both i64::MIN and i64::MAX are present).
let range = ArrayMap::calculate_range(min_val, max_val);// line 156let num_row:usize = batches.iter().map(|x| x.num_rows()).sum();let dense_ratio = (num_row asf64) / ((range + 1)asf64);// line 158 — OVERFLOW// ...if range == usize::MAXasu64{// line 174 — guard is too latereturnOk(None);}
When key_to_u64(i64::MIN) = 9223372036854775808 and key_to_u64(i64::MAX) = 9223372036854775807, calculate_range returns u64::MAX via wrapping_sub. The range + 1 on line 158 then overflows (panic in debug, wrap-to-zero in release → division by zero).
The existing guard on line 174 correctly checks for this case but is placed after the overflow.
Proposed fix
Move the range == usize::MAX as u64 guard before line 158, or replace line 158 with:
Claude helped me write the issue after I found it while testing Comet. Claude also provided the proposed fix, so please double check the latter before opening a PR with that fix. I can look at it in more detail tomorrow if no one else can get to it.
Activity
buraksenn commentedon Mar 17, 2026
take
fix: move overflow guard before dense ratio in hash join to prevent o…
fix: move overflow guard before dense ratio in hash join to prevent o…
[branch-53] fix: move overflow guard before dense ratio in hash join …
fix: move overflow guard before dense ratio in hash join to prevent o…