Make now aware of execution timezone

Is your feature request related to a problem or challenge?

The now() function currently is hardcoded to use +00:00 timezone. Now that functions have access to the ConfigOptions now() can be updated to use the tz set in 'datafusion.execution.time_zone'

Describe the solution you'd like

now() returns a timestamp in the tz set via the 'datafusion.execution.time_zone' config option.

Describe alternatives you've considered

Leave unchanged.

Additional context

No response

Note that currently I don't think this is quite possible because the return_field_from_args function has no access to ConfigOptions. Leaving it as it is results in queries such as

> SET TIME ZONE '+08:00';
0 row(s) fetched. 
Elapsed 0.002 seconds.
> select count(1) result from (select now() as n) a where n = '2000-01-01'::date;
Arrow error: Invalid argument error: Invalid comparison operation: Timestamp(Nanosecond, Some("+08:00")) == Timestamp(Nanosecond, Some("+00:00"))

I've been poking at this for a while and I don't see an easy way to make this work. Wiring ConfigOptions into the return_field_from_args function touches far, far too many things for it to be worth it.

My best thought for making this work is to clone the ScalarUDF whenever it's needed (here for example) to make the UDF stateful and then call a new function say initialize(config_options: Arc<ConfigOptions>) which would then provide the function with any required state it required.

Without something like the above functions like now that do not have a return type that is determined by the args provided but rather determined externally or via configuration just can't work properly.

@Omega359

After investigation, I agree with @Omega359 that wiring ConfigOptions into return_field_from_args would be a massive undertaking with questionable ROI.

However, I'd like us to reconsider whether this feature justifies the architectural complexity.

The Core Issue

The problem manifests in subqueries:

-- Works fine
SELECT now() = '2000-01-01'::date;

-- Fails with schema mismatch
SELECT count(1) FROM (SELECT now() as n) a WHERE n = '2000-01-01'::date;

Root Cause

Subquery schemas are materialized at planning time via return_field_from_args(), which lacks access to ConfigOptions. However, the actual return type comes from simplify(), which does have config access. This creates a schema/data type mismatch that Arrow rightfully rejects.

The Key Question

I don't see many other built-in functions requiring config-dependent return types. If now() is largely an isolated case, a major architectural refactor may not be warranted.

Proposal

Before proceeding with the refactor, I suggest we:

Identify if other functions have similar config-dependent behavior
Evaluate the user impact - how frequently is this pattern encountered?
Consider targeted alternatives (special casing, schema deferral, etc.)

Waiting for others' feedback on whether this pattern is common enough to justify the architectural investment.

The other issues I filed I suspect all would be blocked by this issue:

The other thought I had for this issue was to add a listener for config changes and re-register any udf's that have a new_with_config(config_options: Arc<ConfigOptions>) constructor.

@Omega359

It came up today on our sync call with @Omega359 that this change might make it hard for downstream crates when upgrading to a new version of DataFusion if they can't go back to the existing behavior.

Thus I think we should ensure there is some way (e.g. a configuration setting change) to get the same behavior of DataFusion 50

For example, maybe we could just document that setting the timezone to "None" will return the behavior to DataFusion 50

Update: this is what @Omega359 suggests here: #18017 (comment)

@Weijun-H

Update: @Weijun-H has a PR up for this feature:

fix: Use dynamic timezone in now() function for accurate timestamp #18017

fix: Use dynamic timezone in now() function for accurate timestamp (#…

fix: Use dynamic timezone in now() function for accurate timestamp (a…

Navigation Menu

Repository navigation

Make `now` aware of execution timezone #17993

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Make now aware of execution timezone #17993

Description

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Activity

Omega359 commented on Oct 9, 2025

Omega359 commented on Oct 10, 2025

Weijun-H commented on Oct 12, 2025

Omega359 commented on Oct 13, 2025

alamb commented on Oct 15, 2025

alamb commented on Oct 15, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Make `now` aware of execution timezone #17993