You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Correlated scalar subquery ignores HAVING that is true for empty input #26505
When a correlated scalar subquery aggregates without GROUP BY (for example with sum or max), its HAVING clause is ignored if the predicate is true for an empty input, such as HAVING sum(y.b) IS NULL or HAVING count(*) = 0.
When the HAVING clause should filter out the aggregate row, the subquery returns the aggregate value instead of NULL.
To Reproduce
datafusion-cli -c "SELECT x.a, (SELECT sum(y.b) FROM (VALUES (1, 10)) AS y(a, b) WHERE y.a = x.a HAVING sum(y.b) IS NULL) AS vFROM (VALUES (1), (2)) AS x(a)ORDER BY x.a"
This currently gives:
+---+------+
| a | v |
+---+------+
| 1 | 10 |
| 2 | NULL |
+---+------+
Expected behavior
v should be NULL for both rows, which is what PostgreSQL 17 and SQLite return:
a = 1: sum(y.b) is 10, so sum(y.b) IS NULL is false and the subquery returns no row, giving NULL.
a = 2: there are no matching rows, so sum(y.b) is NULL. The HAVING passes and the subquery returns NULL.
EXPLAIN shows no filter for the HAVING anywhere in the plan.
Additional context
Also happens with max/min, and with HAVING count(*) = 0 while the subquery selects sum(y.b).
Correct results when the subquery is uncorrelated (WHERE y.a = 1), when the HAVING is false for empty input (for example HAVING sum(y.b) > 100), and when the subquery selects count(*) instead (HAVING count(*) = 0 gives NULL, 0).
Reproduced on main (6c1727f, cli v55.1.0): v=10 for a=1 where PostgreSQL 17 returns NULL, and EXPLAIN shows no HAVING filter. Root cause: PullUpCorrelatedExpr pulls a HAVING that is true on empty input out of the subquery and removes the Filter (datafusion/optimizer/src/decorrelate.rs:435), intending to re-apply it via the count-bug compensation CASE. But build_join skips that compensation for aggregates whose empty-input value is NULL (sum/max/min) at datafusion/optimizer/src/scalar_subquery_to_join.rs:407, so the HAVING is never re-applied. count() escapes because its empty-input value 0 is non-NULL, hence HAVING count() = 0 still returns NULL, 0.
When fixing this bug, I think we should make the query error rather than trying to make it work / give correct results. To make it work and have confidence it will be correct, I think we need a more general approach than patches. See here for details
Activity
0xamlab commentedon Oct 21, 2026
Reproduced on main (6c1727f, cli v55.1.0): v=10 for a=1 where PostgreSQL 17 returns NULL, and EXPLAIN shows no HAVING filter. Root cause: PullUpCorrelatedExpr pulls a HAVING that is true on empty input out of the subquery and removes the Filter (datafusion/optimizer/src/decorrelate.rs:435), intending to re-apply it via the count-bug compensation CASE. But build_join skips that compensation for aggregates whose empty-input value is NULL (sum/max/min) at datafusion/optimizer/src/scalar_subquery_to_join.rs:407, so the HAVING is never re-applied. count() escapes because its empty-input value 0 is non-NULL, hence HAVING count() = 0 still returns NULL, 0.
abp002 commentedon Oct 22, 2026
take
alamb commentedon Oct 28, 2026
When fixing this bug, I think we should make the query error rather than trying to make it work / give correct results. To make it work and have confidence it will be correct, I think we need a more general approach than patches. See here for details