You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
During review of #6932 there were several suggstions made about how we could improve the performance of streaming / bounded grouping (when the data is sorted by some/all of the group keys)
It occurs to me that a potentially faster way to detect the group boundaries would be to use the inequality kernel offset by one w.r.t to each other, ORing the results together, and then iterating over the set bits.
There would be some subtlety to handle nulls correctly, but it would likely be significantly faster and would not require converting to the row format
We could likely do something similar for window functions if we aren't already
@alamb I think this issue can be closed. Suggestion 1 is done, suggestion 2 is no longer relevant as there are no hashes computed for GroupOrderingFull
Activity
tustvold commentedon Jul 27, 2023
I implemented suggestion 1 upstream in apache/arrow-rs#4575
smiklos commentedon Aug 11, 2024
@alamb I think this issue can be closed. Suggestion 1 is done, suggestion 2 is no longer relevant as there are no hashes computed for GroupOrderingFull
alamb commentedon Aug 11, 2024
Thanks @smiklos -- done!