Repository navigation
Implement to_unixtime function #5568
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Like from_unixtime in #2871, to_unixtime is sometimes also useful when compared with other data that is in UNIX timestamp format or don't want to care about the timezone.
Describe the solution you'd like
Take to_unix_timestamp in spark as example:
select to_unixtime(ts) from t;
-> 1460041200
Describe alternatives you've considered
Additional context
Activity
comphead commentedon Mar 13, 2023
@waynexia As a workaround please use #5555
@alamb here we comes again to spark functions, should we support all of them like listed here https://spark.apache.org/docs/3.2.0/api/sql/ even if not all of them compatible with PG
For some people spark syntax is more important than PGs, but for others it can be opposite
alamb commentedon Mar 13, 2023
I don't think we have a strong or consistent story here -- my personal opinion is to focus on postgres compatibility, and only add spark functions for thing that have no postgres equivalent.
Given DataFusion has all the extension points already, maybe we could add a separate crate for "spark_extensions" or something 🤔 that way anyone who wanted the spark library functions could have them if they wanted but it could be opt in.
waynexia commentedon Mar 14, 2023
Thanks for this 👍
Also makes sense to me. We can implement and provide it in a separate crate (something like
datafusion-spark-ext?) so that others can easily integrate wanted functions (liketo_unixtimein this ticket) while keeping the core clean. I'd like to help if we decide to do it.alamb commentedon Mar 15, 2023
Cross posting link to discussion on spark functions: #5600
Tangruilin commentedon Jan 29, 2024
@alamb I can help with this.
Please assign to me
alamb commentedon Jan 29, 2024
Thanks @Tangruilin -- you already have several items assigned to you:
https://github.com/apache/arrow-datafusion/issues/assigned/Tangruilin
If there are some that you are not working on, or we found blockers, perhaps you can unassign yourself so it is clear what the status is.
Thank you so much for your help
Tangruilin commentedon Jan 29, 2024
I unassign #8824, other issues i am all working on now.
I will finish them in two weeks.
Add `to_unixtime` function (#9077)