Repository navigation
Implement from_unixtime function #2871
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In the SO post https://stackoverflow.com/questions/72888852/extract-year-month-day-from-unix-timestamp-column-in-rust-datafusion-dataframe/72941102#72941102 the user needed help with translating a unixtime to a timestamp. The solution now is to use a cast and this is verbose and not obvious. It would be nice to have a specific from_unixtime function.
Describe the solution you'd like
See https://spark.apache.org/docs/2.3.0/api/sql/index.html#from_unixtime for Spark's version
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Activity
waitingkuo commentedon Jul 11, 2022
@andygrove
would you mind giving me some hint? I'm still new to this project. thanks~
Is this the right place to begin with https://github.com/apache/arrow-datafusion/blob/master/datafusion/expr/src/built_in_function.rs#L150 ?
comphead commentedon Jul 11, 2022
@waitingkuo https://github.com/apache/arrow-datafusion/blob/master/CONTRIBUTING.md#how-to-add-a-new-scalar-function this can be some start for you
comphead commentedon Jul 11, 2022
#2324 please check this PR also
waitingkuo commentedon Jul 11, 2022
thank you @comphead !
waitingkuo commentedon Jul 16, 2022
@comphead i finally figured out how to add a new function, thank you~
@andygrove
the current implementation only support Int64 as input value, should we support floating number and utf-8 as well?
the pyspark version has the second argument to render user preferred time format, should we support that in this pr?
MrDataPsycho commentedon Oct 4, 2022
@waitingkuo Can you please refer me where you have added that code? I only found 2 references of the function:
But could not find the actual function. I want to add some example usage doc in the function. As it looks empty without any example: crates doc . Or if you can tell how I should add some doc. The contribution guide says the function should be added into datetime expression. But could not find that function there.
waitingkuo commentedon Oct 4, 2022
@DataPsycho
it's located here
https://github.com/apache/arrow-datafusion/blob/45fc415daa7028559ef3477e53a184a114149f9e/datafusion/physical-expr/src/functions.rs#L155-L171
you could follow this to find all related functions
https://github.com/apache/arrow-datafusion/blob/master/docs/source/contributor-guide/index.md#how-to-add-a-new-aggregate-function