Skip to content

Correlated scalar subquery ignores HAVING that is true for empty input #26505

Description

@fornwall

Describe the bug

When a correlated scalar subquery aggregates without GROUP BY (for example with sum or max), its HAVING clause is ignored if the predicate is true for an empty input, such as HAVING sum(y.b) IS NULL or HAVING count(*) = 0.

When the HAVING clause should filter out the aggregate row, the subquery returns the aggregate value instead of NULL.

To Reproduce

datafusion-cli -c "
SELECT x.a,
       (SELECT sum(y.b) FROM (VALUES (1, 10)) AS y(a, b) WHERE y.a = x.a HAVING sum(y.b) IS NULL) AS v
FROM (VALUES (1), (2)) AS x(a)
ORDER BY x.a"

This currently gives:

+---+------+
| a | v    |
+---+------+
| 1 | 10   |
| 2 | NULL |
+---+------+

Expected behavior

v should be NULL for both rows, which is what PostgreSQL 17 and SQLite return:

  • a = 1: sum(y.b) is 10, so sum(y.b) IS NULL is false and the subquery returns no row, giving NULL.
  • a = 2: there are no matching rows, so sum(y.b) is NULL. The HAVING passes and the subquery returns NULL.

EXPLAIN shows no filter for the HAVING anywhere in the plan.

Additional context

Found with DataFusion CLI v55.1.0, main at 7e102d7.

Willingness to contribute

None

Activity

  1. added theissue type on Oct 19, 2026
  2. 0xamlab commented on Oct 21, 2026

    @0xamlab

    Reproduced on main (6c1727f, cli v55.1.0): v=10 for a=1 where PostgreSQL 17 returns NULL, and EXPLAIN shows no HAVING filter. Root cause: PullUpCorrelatedExpr pulls a HAVING that is true on empty input out of the subquery and removes the Filter (datafusion/optimizer/src/decorrelate.rs:435), intending to re-apply it via the count-bug compensation CASE. But build_join skips that compensation for aggregates whose empty-input value is NULL (sum/max/min) at datafusion/optimizer/src/scalar_subquery_to_join.rs:407, so the HAVING is never re-applied. count() escapes because its empty-input value 0 is non-NULL, hence HAVING count() = 0 still returns NULL, 0.

  3. abp002 commented on Oct 22, 2026

    @abp002
    Contributor

    take

  4. alamb commented on Oct 28, 2026

    @alamb
    Contributor

    When fixing this bug, I think we should make the query error rather than trying to make it work / give correct results. To make it work and have confidence it will be correct, I think we need a more general approach than patches. See here for details

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions