Repository navigation
UNION ALL panics on wasm32-unknown-unknown #24886
Description
Describe the bug
DataFusion compiled to wasm32-unknown-unknown panics on UNION ALL and any other multi-partition query whose collect funnels through CoalescePartitionsExec (aggregations over partitioned scans, sorts across partitions, etc.).
CoalescePartitionsExec::execute spawns one task per input partition via JoinSet::spawn on the _ => (multi-input) branch. JoinSet::spawn needs a tokio reactor that isn't installed under wasm-bindgen-futures. The panic:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
The single-input branch of CoalescePartitionsExec::execute works — it passes the child stream through directly. Only the multi-input branch spawns.
Follow-up to #24275, which fixed the same class of panic at a different call site (collect_partitioned).
To Reproduce
Add this to datafusion/wasmtest/src/lib.rs:
#[wasm_bindgen_test(unsupported = tokio::test)]
async fn test_union_all() {
let ctx = get_ctx();
ctx.sql("SELECT 1 AS n UNION ALL SELECT 2 AS n")
.await.unwrap()
.collect().await.unwrap();
}Run from the repo root:
RUSTFLAGS='--cfg getrandom_backend="wasm_js"' \
wasm-pack test --headless --chrome datafusion/wasmtestThe test panics:
panicked at datafusion/common-runtime/src/join_set.rs:69:20:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
Expected behavior
Multi-partition collect should succeed under wasm-bindgen-futures when the session has asked for single-threaded execution (target_partitions == 1). Sequential drain of the input partitions is a valid interleaving — CoalescePartitionsExec's own docstring states "No guarantees are made about the order of the resulting partition."
Activity
yinli-systems commentedon Sep 2, 2026
take
kentkwu commentedon Sep 2, 2026
@yinli-systems sorry I didn't see your comment before I put up the pr
yinli-systems commentedon Sep 2, 2026
untake
kentkwu commentedon Sep 2, 2026
take
tschwarzinger commentedon Sep 14, 2026
I am not sure whether this is strictly necessary. You can spawn a single-threaded tokio runtime in wasm and execute all DataFusion quries inside it. At least that's what we're trying to do and it seems to work:
Feel free to try it out yourself:
https://tschwarzinger.codeberg.page/rdf-fusion/playground/
In the engine settings you can set the target partitions. It's sparql though, not sql so I am not sure how easy you can test your query. Be sure to use the latest nightly build.
You can take a look at the code here:
https://codeberg.org/tschwarzinger/rdf-fusion/src/branch/main/lib/wasm/src/runtime.rs
It is still in PoC state though, not really production ready I gather.
EDIT: I am not sure how well the current design will yield to the browser event loop if interactivity is necessary. We run the queries in a web worker.
fix(physical-plan): CoalescePartitionsExec panic on wasm32-unknown-un…
fix(physical-plan): CoalescePartitionsExec panic on wasm32-unknown-un…