Repository navigation
Convert BuiltInWindowFunction::{Rank, PercentRank, DenseRank} to a user defined functions #12648
Description
Is your feature request related to a problem or challenge?
Part of #8709
There is now no difference between "built in" / "prepackaged" scalar and aggregate functions in DataFusion, however there are still some "built in" window functions -- see the current source for BuiltInWindowFunction for the up to date list of what remains
The problem with having two different kinds of window functions is
- There are some features that may not be available to User Defined Window Functions that rely on built in
- Users can not easily choose which window functions to include or override the behavior if they need something different
Describe the solution you'd like
I would like to remove the "built in" version of this function and convert it to a user defined function
Describe alternatives you've considered
At a high level the process is:
- Add a new WindowUDFImpl in the
functions-windowcrate - Port the code from the relevant existing implementation of the the built in functions in datafusion/physical-expr/src/window
- Remove the
BuiltInWindowFunctionvariant and then get everything to compile (the compiler will show you where the existing implementations are)
Additional context
Here are some good examples:
- @jcsherin ported
RowNumberin Convert built-inrow_numberto user-defined window function #12030
Activity
alamb commentedon Sep 27, 2024
I think this is a good first issue for someone skilled in rust that wants to get a handle on how datafusion works. If you hit issues, let us know
dharanad commentedon Sep 27, 2024
takeI think i should give up on this, let someone else who is new to this project take this up. Should be a good learning exp for them.
hailelagi commentedon Sep 27, 2024
take (unless this might be intended for someone else, happy to step back)
jatin510 commentedon Sep 28, 2024
If you don't mind . Can i work on this issue @hailelagi ?
jatin510 commentedon Sep 28, 2024
take
Omega359 commentedon Sep 28, 2024
Convert BuiltInScalarFunction:: -> Convert BuiltInWindowFunction::{NthValue}
[-]Convert `BuiltInScalarFunction::{Rank, PercentRank, DenseRank}` to a user defined functions[/-][+]Convert `BuiltInWindowFunction::{Rank, PercentRank, DenseRank}` to a user defined functions[/+]SteNicholas commentedon Sep 30, 2024
@jatin510, are you working on this ticket? I'm new comer to contribute. Could this ticket assign to me? cc @alamb
jatin510 commentedon Sep 30, 2024
Thanks for reaching out. I’m actually already working on this issue, and it’s my first task in the project as well. I’d like to continue with it. @SteNicholas
jcsherin commentedon Sep 30, 2024
@SteNicholas Will you be interested in picking up
ntilewindow function? It is also pending conversion to a user-defined window function and looks like a good first issue to me.See https://github.com/apache/datafusion/blob/main/datafusion/physical-expr/src/window/ntile.rs.
migrate window functions rank, dense_rank, and percent_rank to udwf
migrate window functions rank, dense_rank, and percent_rank to udwf
Upgrade to Datafusion 43 (#905)