Skip to content

multiply overflow in stats.rs #13775

New issue
New issue
Open
Open
multiply overflow in stats.rs#13775
Assignees
LindaSummer
Labels
bugSomething isn't working

Description

@Omega359
Omega359
opened on Dec 13, 2024

Describe the bug

Seeing this when testing the new sqlite tests in sqllogictest against main. It occurs with the select4.slt test file only. backtrace below:

thread 'tokio-runtime-worker' panicked at datafusion/common/src/stats.rs:151:84:
attempt to multiply with overflow
stack backtrace:
   0: rust_begin_unwind
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/std/src/panicking.rs:662:5
   1: core::panicking::panic_fmt
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/panicking.rs:74:14
   2: core::panicking::panic_const::panic_const_mul_overflow
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/panicking.rs:181:21
   3: <usize as core::ops::arith::Mul>::mul
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/ops/arith.rs:342:45
   4: <&usize as core::ops::arith::Mul<&usize>>::mul
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/internal_macros.rs:58:17
   5: datafusion_common::stats::Precision<usize>::multiply
             at /apache_datafusion/datafusion/common/src/stats.rs:151:84
   6: datafusion_physical_plan::joins::cross_join::stats_cartesian_product
             at /apache_datafusion/datafusion/physical-plan/src/joins/cross_join.rs:340:27
   7: <datafusion_physical_plan::joins::cross_join::CrossJoinExec as datafusion_physical_plan::execution_plan::ExecutionPlan>::statistics
             at /apache_datafusion/datafusion/physical-plan/src/joins/cross_join.rs:322:12
   8: datafusion::physical_optimizer::enforce_distribution::get_repartition_requirement_status
             at /apache_datafusion/datafusion/core/src/physical_optimizer/enforce_distribution.rs:1103:49
   9: datafusion::physical_optimizer::enforce_distribution::ensure_distribution
             at /apache_datafusion/datafusion/core/src/physical_optimizer/enforce_distribution.rs:1197:9
  10: <datafusion::physical_optimizer::enforce_distribution::EnforceDistribution as datafusion_physical_optimizer::optimizer::PhysicalOptimizerRule>::optimize::{{closure}}
             at /apache_datafusion/datafusion/core/src/physical_optimizer/enforce_distribution.rs:217:17
  11: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/ops/function.rs:305:13
  12: datafusion_common::tree_node::Transformed<T>::transform_parent
             at /apache_datafusion/datafusion/common/src/tree_node.rs:764:44
  13: datafusion_common::tree_node::TreeNode::transform_up::transform_up_impl::{{closure}}
             at /apache_datafusion/datafusion/common/src/tree_node.rs:265:13
  14: stacker::maybe_grow
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/stacker-0.1.17/src/lib.rs:55:9
  15: datafusion_common::tree_node::TreeNode::transform_up::transform_up_impl
             at /apache_datafusion/datafusion/common/src/tree_node.rs:260:9
  16: datafusion_common::tree_node::TreeNode::transform_up::transform_up_impl::{{closure}}::{{closure}}
             at /apache_datafusion/datafusion/common/src/tree_node.rs:265:35
  17: <I as datafusion_common::tree_node::TreeNodeIterator>::map_until_stop_and_collect::{{closure}}
             at /apache_datafusion/datafusion/common/src/tree_node.rs:1125:17
  18: core::iter::adapters::map::map_try_fold::{{closure}}
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/map.rs:95:28
  19: <alloc::vec::into_iter::IntoIter<T,A> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/into_iter.rs:346:25
  20: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/map.rs:121:9
  21: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/mod.rs:191:9
  22: <I as alloc::vec::in_place_collect::SpecInPlaceCollect<T,I>>::collect_in_place
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/in_place_collect.rs:378:13
  23: alloc::vec::in_place_collect::from_iter_in_place
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/in_place_collect.rs:269:9
  24: alloc::vec::in_place_collect::<impl alloc::vec::spec_from_iter::SpecFromIter<T,I> for alloc::vec::Vec<T>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/in_place_collect.rs:245:9
  25: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/mod.rs:2985:9
  26: core::iter::traits::iterator::Iterator::collect
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/traits/iterator.rs:2000:9
  27: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/result.rs:1958:51
  28: core::iter::adapters::try_process
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/mod.rs:160:17
  29: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/result.rs:1958:9
  30: core::iter::traits::iterator::Iterator::collect
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/traits/iterator.rs:2000:9
  31: <I as datafusion_common::tree_node::TreeNodeIterator>::map_until_stop_and_collect
             at /apache_datafusion/datafusion/common/src/tree_node.rs:1123:9
  32: <T as datafusion_common::tree_node::TreeNode>::map_children
             at /apache_datafusion/datafusion/common/src/tree_node.rs:1256:32
  33: datafusion_common::tree_node::TreeNode::transform_up::transform_up_impl::{{closure}}
             at /apache_datafusion/datafusion/common/src/tree_node.rs:265:13
  34: stacker::maybe_grow
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/stacker-0.1.17/src/lib.rs:55:9
  35: datafusion_common::tree_node::TreeNode::transform_up::transform_up_impl
             at /apache_datafusion/datafusion/common/src/tree_node.rs:260:9
  36: datafusion_common::tree_node::TreeNode::transform_up::transform_up_impl::{{closure}}::{{closure}}
             at /apache_datafusion/datafusion/common/src/tree_node.rs:265:35
  37: <I as datafusion_common::tree_node::TreeNodeIterator>::map_until_stop_and_collect::{{closure}}
             at /apache_datafusion/datafusion/common/src/tree_node.rs:1125:17
  38: core::iter::adapters::map::map_try_fold::{{closure}}
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/map.rs:95:28
  39: <alloc::vec::into_iter::IntoIter<T,A> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/into_iter.rs:346:25
  40: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/map.rs:121:9
  41: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/mod.rs:191:9
  42: <I as alloc::vec::in_place_collect::SpecInPlaceCollect<T,I>>::collect_in_place
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/in_place_collect.rs:378:13
  43: alloc::vec::in_place_collect::from_iter_in_place
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/in_place_collect.rs:269:9
  44: alloc::vec::in_place_collect::<impl alloc::vec::spec_from_iter::SpecFromIter<T,I> for alloc::vec::Vec<T>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/in_place_collect.rs:245:9
  45: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/mod.rs:2985:9
  46: core::iter::traits::iterator::Iterator::collect
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/traits/iterator.rs:2000:9
  47: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/result.rs:1958:51
  48: core::iter::adapters::try_process
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/mod.rs:160:17
  49: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/result.rs:1958:9
  50: core::iter::traits::iterator::Iterator::collect
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/traits/iterator.rs:2000:9
  51: <I as datafusion_common::tree_node::TreeNodeIterator>::map_until_stop_and_collect
             at /apache_datafusion/datafusion/common/src/tree_node.rs:1123:9
  52: <T as datafusion_common::tree_node::TreeNode>::map_children
             at /apache_datafusion/datafusion/common/src/tree_node.rs:1256:32
  53: datafusion_common::tree_node::TreeNode::transform_up::transform_up_impl::{{closure}}
             at /apache_datafusion/datafusion/common/src/tree_node.rs:265:13
  54: stacker::maybe_grow
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/stacker-0.1.17/src/lib.rs:55:9
  55: datafusion_common::tree_node::TreeNode::transform_up::transform_up_impl
             at /apache_datafusion/datafusion/common/src/tree_node.rs:260:9
  56: datafusion_common::tree_node::TreeNode::transform_up
             at /apache_datafusion/datafusion/common/src/tree_node.rs:269:9
  57: <datafusion::physical_optimizer::enforce_distribution::EnforceDistribution as datafusion_physical_optimizer::optimizer::PhysicalOptimizerRule>::optimize
             at /apache_datafusion/datafusion/core/src/physical_optimizer/enforce_distribution.rs:215:36
  58: datafusion::physical_planner::DefaultPhysicalPlanner::optimize_physical_plan
             at /apache_datafusion/datafusion/core/src/physical_planner.rs:1878:24
  59: <datafusion::physical_planner::DefaultPhysicalPlanner as datafusion::physical_planner::PhysicalPlanner>::create_physical_plan::{{closure}}
             at /apache_datafusion/datafusion/core/src/physical_planner.rs:184:17
  60: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/future/future.rs:123:9
  61: <datafusion::execution::session_state::DefaultQueryPlanner as datafusion::execution::context::QueryPlanner>::create_physical_plan::{{closure}}
             at /apache_datafusion/datafusion/core/src/execution/session_state.rs:1943:14
  62: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/future/future.rs:123:9
  63: datafusion::execution::session_state::SessionState::create_physical_plan::{{closure}}
             at /apache_datafusion/datafusion/core/src/execution/session_state.rs:735:14
  64: datafusion::dataframe::DataFrame::create_physical_plan::{{closure}}
             at /apache_datafusion/datafusion/core/src/dataframe/mod.rs:219:61
  65: datafusion_sqllogictest::engines::datafusion_engine::runner::run_query::{{closure}}
             at ./src/engines/datafusion_engine/runner.rs:91:42
  66: <datafusion_sqllogictest::engines::datafusion_engine::runner::DataFusion as sqllogictest::runner::AsyncDB>::run::{{closure}}
             at ./src/engines/datafusion_engine/runner.rs:60:48
  67: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/future/future.rs:123:9
  68: sqllogictest::runner::Runner<D,M>::apply_record::{{closure}}
             at /usr/local/cargo/git/checkouts/sqllogictest-rs-417ef58aa74844f4/98ea24e/sqllogictest/src/runner.rs:773:62
  69: sqllogictest::runner::Runner<D,M>::update_test_file::{{closure}}
             at /usr/local/cargo/git/checkouts/sqllogictest-rs-417ef58aa74844f4/98ea24e/sqllogictest/src/runner.rs:1412:75
  70: sqllogictests::run_complete_file::{{closure}}
             at ./bin/sqllogictests.rs:367:10
  71: sqllogictests::run_tests::{{closure}}::{{closure}}::{{closure}}
             at ./bin/sqllogictests.rs:208:83

To Reproduce

I can reproduce repeatable in my branch @ https://github.com/Omega359/arrow-datafusion/tree/feature/sqllogictest_add_sqlite when running the sqlite complete

cargo test --features postgres --test sqllogictests -- --complete --postgres-runner --include-sqlite

It does seem to be any particular sql in that file that is causing the issue but rather the complete number of them.

Correction - it's always failing on a particular sql however running the equivalent sql in datafusion-cli does not cause the issue. That is possibly because of a difference in how things are run - in sqllogictests the DF queries are run via:

async fn run_query(ctx: &SessionContext, sql: impl Into<String>) -> Result<DFOutput> {
    let df = ctx.sql(sql.into().as_str()).await?;
    let task_ctx = Arc::new(df.task_ctx());
    let plan = df.create_physical_plan().await?; 
#   ^-- the above line is triggering the code that panics

    let stream = execute_stream(plan, task_ctx)?;
    let types = normalize::convert_schema_to_types(stream.schema().fields());
    let results: Vec<RecordBatch> = collect(stream).await?;
    let rows = normalize::convert_batches(results)?;

Expected behavior

No overflow :)

Additional context

No response

Activity

  1. Omega359
    added
    bugSomething isn't working
    on Dec 13, 2024
  2. alamb
    changed the title [-]attempt to multiple with overflow in stats.rs[/-] [+]attempt to multiply with overflow in stats.rs[/+] on Dec 16, 2024
  3. alamb
    changed the title [-]attempt to multiply with overflow in stats.rs[/-] [+]multiply overflow in stats.rs[/+] on Dec 17, 2024

  4. LindaSummer commented on Jan 7, 2025

    @LindaSummer
    LindaSummer
    on Jan 7, 2025
    Member

    Hi @Omega359 ,

    I'm a newbie of datafusion and would like to try to make some contribution. 😊

    Could this issue be assigned to me?

    Best Regards,
    Edward

  5. Omega359 commented on Jan 7, 2025

    @Omega359
    Omega359
    on Jan 7, 2025
    ContributorAuthor

    Hi Edward - Thanks for taking a look at this! I do not have the ability to do that however if you add a comment here with just the word 'take' a github action will assign it to you

  6. LindaSummer commented on Jan 7, 2025

    @LindaSummer
    LindaSummer
    on Jan 7, 2025
    Member

    Hi Edward - Thanks for taking a look at this! I do not have the ability to do that however if you add a comment here with just the word 'take' a github action will assign it to you

    Got it!

    Thanks very much! 😊

    Best Regards,
    Edward

  7. LindaSummer commented on Jan 7, 2025

    @LindaSummer
    LindaSummer
    on Jan 7, 2025
    Member

    take

  8. github-actions
    assigned
    LindaSummer
    on Jan 7, 2025

  9. LindaSummer commented on Jan 29, 2025

    @LindaSummer
    LindaSummer
    on Jan 29, 2025
    Member

    Hi,

    Sorry for delay on this issue.

    I will try to work on it now. 😊

    Best Regards,
    Edward

  10. LindaSummer commented on Feb 13, 2025

    @LindaSummer
    LindaSummer
    on Feb 13, 2025
    Member

    Hi @Omega359 ,

    Sorry to delay this issue and bother you again. 😊

    I find that the sqllogictest_add_sqlite branch seems to have been merged in #13936.
    So, I ran cargo test --features postgres --test sqllogictests -- --complete --postgres-runner --include-sqlite in current master branch and find that it didn't print any error message.

    Does this issue still exist, or my reproducing steps are incorrect?

    Best Regards,
    Edward

  11. Omega359 commented on Feb 13, 2025

    @Omega359
    Omega359
    on Feb 13, 2025 via email
    ContributorAuthor

  12. LindaSummer commented on Feb 15, 2025

    @LindaSummer
    LindaSummer
    on Feb 15, 2025
    Member

    Hi @Omega359 ,

    Sorry to interrupt you. 😊

    I used the commit in https://github.com/Omega359/arrow-datafusion/tree/sqllogictest_with_sqlite in Nov 2024.
    But it didn't print error message.

    Would you mind give me a commit that can trigger this problem?

    Thanks very much.

    Best Regards,
    Edward

  13. Omega359 commented on Feb 15, 2025

    @Omega359
    Omega359
    on Feb 15, 2025
    ContributorAuthor

    I don't think there was a commit where I saw it - it happened during my local dev of the sqlite testing.

  14. LindaSummer commented on Feb 16, 2025

    @LindaSummer
    LindaSummer
    on Feb 16, 2025
    Member

    I don't think there was a commit where I saw it - it happened during my local dev of the sqlite testing.

    Got it! Thanks very much for your warm help! I will try to solve it. 😄

  15. Speculative commented on Mar 21, 2025

    @Speculative
    Speculative
    on Mar 21, 2025

    I'm reproducing this multiplication overflow panic as well.

    Details

    datafusion version 46.0.0

    Trying to execute Join Order Benchmark query 16B

    If it matters, I've loaded the relevant tables from parquet files converted from the Join Order Benchmark's original data dump.

    Code

        let state = ctx.state();
    let logical_plan = state.create_logical_plan(JOB_16B).await?;

    let planner = DefaultPhysicalPlanner::default();
    let physical_plan = planner.create_physical_plan(&logical_plan, &state).await?;

    Notably, this panic does not reproduce when simply calling ctx.sql(JOB_16B), but my use case requires inspecting the generated physical plan.

    Investigation

    Looks like it comes from calculating stats for one side of a join. This query joins many large-ish tables. Since this plan isn't optimized, it's a bunch of cross joins of big tables.

    Backtrace

    thread 'main' panicked at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-common-46.0.0/src/stats.rs:148:76:
attempt to multiply with overflow
stack backtrace:
   0: rust_begin_unwind
             at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/std/src/panicking.rs:692:5
   1: core::panicking::panic_fmt
             at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/core/src/panicking.rs:75:14
   2: core::panicking::panic_const::panic_const_mul_overflow
             at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/core/src/panicking.rs:178:21
   3: <usize as core::ops::arith::Mul>::mul
             at /home/jtao/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/arith.rs:355:45
   4: <&usize as core::ops::arith::Mul>::mul
             at /home/jtao/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/internal_macros.rs:58:17
   5: datafusion_common::stats::Precision<usize>::multiply
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-common-46.0.0/src/stats.rs:148:76
   6: datafusion_physical_plan::joins::cross_join::stats_cartesian_product
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/joins/cross_join.rs:396:27
   7: <datafusion_physical_plan::joins::cross_join::CrossJoinExec as datafusion_physical_plan::execution_plan::ExecutionPlan>::statistics
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/joins/cross_join.rs:337:12
   8: <datafusion_physical_plan::joins::cross_join::CrossJoinExec as datafusion_physical_plan::execution_plan::ExecutionPlan>::statistics
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/joins/cross_join.rs:338:13
   9: <datafusion_physical_plan::joins::cross_join::CrossJoinExec as datafusion_physical_plan::execution_plan::ExecutionPlan>::statistics
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/joins/cross_join.rs:338:13
  10: <datafusion_physical_plan::joins::cross_join::CrossJoinExec as datafusion_physical_plan::execution_plan::ExecutionPlan>::statistics
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/joins/cross_join.rs:338:13
  11: <datafusion_physical_plan::joins::cross_join::CrossJoinExec as datafusion_physical_plan::execution_plan::ExecutionPlan>::statistics
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/joins/cross_join.rs:338:13
  12: <datafusion_physical_plan::joins::cross_join::CrossJoinExec as datafusion_physical_plan::execution_plan::ExecutionPlan>::statistics
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/joins/cross_join.rs:338:13
  13: datafusion_physical_plan::filter::FilterExec::statistics_helper
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/filter.rs:176:27
  14: datafusion_physical_plan::filter::FilterExec::compute_properties
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/filter.rs:258:21
  15: datafusion_physical_plan::filter::FilterExec::try_new
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-physical-plan-46.0.0/src/filter.rs:87:29
  16: datafusion::physical_planner::DefaultPhysicalPlanner::map_logical_node_to_physical::{{closure}}
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-46.0.0/src/physical_planner.rs:779:30
  17: datafusion::physical_planner::DefaultPhysicalPlanner::task_helper::{{closure}}
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-46.0.0/src/physical_planner.rs:389:26
  18: <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as futures_core::stream::Stream>::poll_next
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/futures_unordered/mod.rs:528:17
  19: futures_util::stream::stream::StreamExt::poll_next_unpin
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/stream/mod.rs:1638:9
  20: <futures_util::stream::stream::buffer_unordered::BufferUnordered<St> as futures_core::stream::Stream>::poll_next
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/stream/buffer_unordered.rs:75:15
  21: <S as futures_core::stream::TryStream>::try_poll_next
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-core-0.3.31/src/stream.rs:206:9
  22: <futures_util::stream::try_stream::try_collect::TryCollect<St,C> as core::future::future::Future>::poll
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/futures-util-0.3.31/src/stream/try_stream/try_collect.rs:46:26
  23: datafusion::physical_planner::DefaultPhysicalPlanner::create_initial_plan::{{closure}}
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-46.0.0/src/physical_planner.rs:335:14
  24: <datafusion::physical_planner::DefaultPhysicalPlanner as datafusion::physical_planner::PhysicalPlanner>::create_physical_plan::{{closure}}
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/datafusion-46.0.0/src/physical_planner.rs:183:22
  25: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /home/jtao/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9
  26: datafusion_oracle::main::{{closure}}
             at ./src/main.rs:59:77
  27: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /home/jtao/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:124:9
  28: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/park.rs:284:60
  29: tokio::task::coop::with_budget
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/task/coop/mod.rs:167:5
  30: tokio::task::coop::budget
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/task/coop/mod.rs:133:5
  31: tokio::runtime::park::CachedParkThread::block_on
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/park.rs:284:31
  32: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/context/blocking.rs:66:9
  33: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  34: tokio::runtime::context::runtime::enter_runtime
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/context/runtime.rs:65:16
  35: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  36: tokio::runtime::runtime::Runtime::block_on_inner
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/runtime.rs:370:45
  37: tokio::runtime::runtime::Runtime::block_on
             at /home/jtao/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/runtime.rs:340:13
  38: datafusion_oracle::main
             at ./src/main.rs:63:5
  39: core::ops::function::FnOnce::call_once
             at /home/jtao/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

  16. LindaSummer commented on Mar 23, 2025

    @LindaSummer
    LindaSummer
    on Mar 23, 2025
    Member

    Hi @Speculative ,

    Thanks very much for your investigation! ❤

    I'm sorry that in last month due to my personal circumstances, I didn't have enough time to follow up on this issue.

    I will try to follow your reproducing steps for this problem.

    I will try to update in this thread or reach out to you when I have any new progress or problem on reproducing if you don't mind.

    Best Regards,
    Edward

  17. Speculative commented on Mar 26, 2025

    @Speculative
    Speculative
    on Mar 26, 2025

    In my case, after comparing my implementation to what's happening in ctx.sql(), I realized that I was missing a line to optimize the logical plan, so this panics:

    let state = ctx.state();
let logical_plan = state.create_logical_plan(JOB_16B).await?;

let planner = DefaultPhysicalPlanner::default();
let physical_plan = planner.create_physical_plan(&logical_plan, &state).await?;

    But this does not:

    let state = ctx.state();
let logical_plan = state.create_logical_plan(JOB_16B).await?;
let opt_logical_plan = state.optimize(&logical_plan);

let planner = DefaultPhysicalPlanner::default();
let physical_plan = planner.create_physical_plan(&logical_plan, &state).await?;

    Which is also equivalent to:

    let state = ctx.state();
let logical_plan = state.create_logical_plan(JOB_16B).await?;
let physical_plan = state.create_physical_plan(&logical_plan).await?;

  18. LindaSummer commented on Mar 27, 2025

    @LindaSummer
    LindaSummer
    on Mar 27, 2025
    Member

    Hi @Speculative ,

    Thanks so much for your investigation and example code! ❤

    With the reproduction case, it will be very helpful for root cause analysis.

    I will try my best to solve this issue.

    Best Regards,
    Edward

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions