Skip to content

date_part declares a non-nullable result for non-null input, but can return NULL #26685

Closed

Description

@mbutrovich

Describe the bug

date_part fails when a non-null input gives a NULL result:

SELECT extract(second from ts)
FROM (VALUES (arrow_cast(9223372036854775807, 'Timestamp(Second, None)'))) AS t(ts);
-- Arrow error: Invalid argument error: Column 'date_part(Utf8("SECOND"),t.ts)' is declared as non-nullable but contains null values

SELECT extract(hour from d)
FROM (VALUES (arrow_cast(9223372036854775807, 'Duration(Second)'))) AS t(d);
-- Arrow error: Invalid argument error: Column 'date_part(Utf8("HOUR"),t.d)' is declared as non-nullable but contains null values

To Reproduce

Run the queries above on main (dad9063be).

Expected behavior

Both return NULL.

Additional context

return_field_from_args takes the result's nullability from the input. Arrow's date_part returns NULL for some non-null inputs, such as timestamps it can't convert and values that overflow Int32 (Duration(s) hour).

The result was always nullable until #19823 (53.0.0). That PR added the Spark date_part, which simplifies to this function, and changed the nullability so the two fields would match (review thread). The Spark wrapper still takes its nullability from the inputs, so a fix here should change both to keep them matching. The optimizer's schema check (logically_equivalent_names_and_types) compares names and types but not nullability, so a mismatch between them would go unnoticed.

Part of #26672.

Willingness to contribute

I plan to contribute a fix for this bug independently.

Activity

  1. self-assigned this
    on Oct 23, 2026
  2. added theissue type on Oct 23, 2026
  3. added a commit that references this issue on Oct 25, 2026
    8ea0dbf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions