sqllogictests fail in debug mode (join_full_multi_batch::batch_size_4_2)

Describe the bug

It fails with

thread 'tokio-runtime-worker' (29848) panicked at /Users/danielheres/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-select-58.0.0/src/coalesce/primitive.rs:61:9:
assertion `left == right` failed
  left: 4
 right: 2
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/std/src/panicking.rs:689:5
   1: core::panicking::panic_fmt
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:80:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:394:5
   4: arrow_select::coalesce::primitive::InProgressPrimitiveArray<T>::ensure_capacity
             at /Users/danielheres/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-select-58.0.0/src/coalesce/primitive.rs:61:9
   5: <arrow_select::coalesce::primitive::InProgressPrimitiveArray<T> as arrow_select::coalesce::InProgressArray>::copy_rows
             at /Users/danielheres/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-select-58.0.0/src/coalesce/primitive.rs:71:14

To Reproduce

No response

Expected behavior

No response

Additional context

No response

A work around is to run it like

cargo test --profile=ci --test sqllogictests

We should fix it for real

take

After investigating this i found out, that the root cause is a faulty debug_assert_eq! in arrow-select's BatchCoalescer that assumes Vec::reserve(n) gives exactly n capacity, which Rust doesn't guarantee.
Filed an upstream issue: apache/arrow-rs#9506

Not sure what the project's approach is for the fix here, but since it's not mission critical
(debug mode only, workaround exists via --profile=ci) it should be okay to wait for the
upstream fix.

Edit: Arrow PR is open: apache/arrow-rs#9508

Not sure what the project's approach is for the fix here, but since it's not mission critical (debug mode only, workaround exists via --profile=ci) it should be okay to wait for the upstream fix.

You can also do RUSTFLAGS="-C debug-assertions=no" (I'm hitting this right now as well).

The fix has been merged in apache/arrow-rs#9506. This should be resolved once DataFusion upgrades to the next Arrow release.

This also came while trying to validate the 53 release:

failures:

----
joins::hash_join::exec::tests::join_full_multi_batch::batch_size_4_2::use_perfect_hash_join_as_possible_1_true
stdout ----

thread
'joins::hash_join::exec::tests::join_full_multi_batch::batch_size_4_2::use_perfect_hash_join_as_possible_1_true'
(1910718) panicked at
/var/folders/wq/mxmvfjtd1kjcrm3bdfrn2zd80000gn/T/datafusion-53.0.0.XXXXX.5PoUgcHv4I/apache-datafusion-53.0.0/test-rustup/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-select-58.0.0/src/coalesce/primitive.rs:61:9:
assertion `left == right` failed
  left: 4
 right: 2

----
joins::hash_join::exec::tests::join_full_multi_batch::batch_size_4_2::use_perfect_hash_join_as_possible_2_false
stdout ----

thread
'joins::hash_join::exec::tests::join_full_multi_batch::batch_size_4_2::use_perfect_hash_join_as_possible_2_false'
(1910719) panicked at
/var/folders/wq/mxmvfjtd1kjcrm3bdfrn2zd80000gn/T/datafusion-53.0.0.XXXXX.5PoUgcHv4I/apache-datafusion-53.0.0/test-rustup/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-select-58.0.0/src/coalesce/primitive.rs:61:9:
assertion `left == right` failed
  left: 4
 right: 2

fix: Run release verification with `--profile=ci` (#20987)

fix: Run release verification with `--profile=ci` (apache#20987)

After a somewhat unfortunate delay, here is a PR to update us to arrow 58.1.0 (that fixes the problem running tests in debug mode)

Update to arrow/parquet 58.1.0 #21044

Update to arrow/parquet `58.1.0` (#21044)

Navigation Menu

Repository navigation

sqllogictests fail in debug mode (join_full_multi_batch::batch_size_4_2) #20689

Describe the bug

To Reproduce

Expected behavior

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

sqllogictests fail in debug mode (join_full_multi_batch::batch_size_4_2) #20689

Description

Describe the bug

To Reproduce

Expected behavior

Additional context

Activity

alamb commented on Mar 4, 2026

Tim-53 commented on Mar 5, 2026

Tim-53 commented on Mar 5, 2026

mbutrovich commented on Mar 6, 2026

Tim-53 commented on Mar 9, 2026

alamb commented on Mar 17, 2026

alamb commented on Mar 24, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions