Skip to content

corr, covar_pop, covar_samp and regr_* silently ignore DISTINCT #26471

Closed

Description

@jayzhan211

Describe the bug

corr, covar_pop, covar_samp and the regr_* functions return the same result with DISTINCT as without it. The planner accepts the query, but the accumulators never read is_distinct.

To Reproduce

SELECT regr_count(x, y)          AS rc,
       regr_count(DISTINCT x, y) AS rc_d,
       covar_pop(x, y)           AS cv,
       covar_pop(DISTINCT x, y)  AS cv_d
FROM (VALUES (1.0, 1.0), (1.0, 1.0), (2.0, 3.0)) t(x, y);

DataFusion returns rc = 3, rc_d = 3, cv = 0.444..., cv_d = 0.444....

Expected behavior

DISTINCT applies to the argument tuple: rc_d = 2, cv_d = 0.5. DuckDB returns these values. If DataFusion does not support it, the query should fail with a clear error instead of returning the non-distinct answer.

Additional context

#25288 tagged these functions with DistinctHandling::Unsupported, and the code says so directly: "the accumulator does not read is_distinct and today silently returns the non-distinct answer. The tag records the intent; enforcement is a follow-up change."
https://github.com/apache/datafusion/blob/3470bd0fbc/datafusion/functions-aggregate/src/covariance.rs#L132-L137

I couldn't find an issue tracking that follow-up. Both options look reasonable: reject f(DISTINCT ..) at planning time when the function reports Unsupported, or deduplicate the input for it. Rejecting is the smaller change, and it's at least correct.

Related: #25374 and #25375 (forwarding distinct_handling).

Willingness to contribute

Feel free to take it!

Activity

  1. abp002 commented on Oct 18, 2026

    @abp002
    Contributor

    take

  2. added a commit that references this issue on Oct 22, 2026
    61ec845
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions