Repository navigation
Stop copying LogicalPlan and Exprs in SingleDistinctToGroupBy #10295
Description
Is your feature request related to a problem or challenge?
Part of #9637
As part of making the planner faster, we are updating the optimizer passes to avoid copying LogicalPlan and Expr (see #9637)
Describe the solution you'd like
I would like to reduce the amount of copying in this pass (even though it doesn't appear in current profiling)
Describe alternatives you've considered
Apply the model from @Lordworms in #10166 to this pass
2. Update OptimizerRule::supports_rewrite` to return true
- Update
OptimizerRuleto userewrite - Update the pass itself to not copy the LogicalPlan (ideally using the
TreeNodeAPI) - it is implemented forLogicalPlan(API) andExpr(API)
Other examples: #10218
Additional context
I don't think this pass does much copying at the moment, for whwat it is woth -- the benfit would be to follow the same pattern
Metadata
Metadata
Assignees
Labels
No labels
Activity
appletreeisyellow commentedon May 13, 2024
I'd like to take this one if no one has worked on it yet
alamb commentedon May 13, 2024
Thanks @appletreeisyellow -- that would be great. No one has started as far as I can tell.
alamb commentedon May 13, 2024
I took a quick look at
datafusion/datafusion/optimizer/src/single_distinct_to_groupby.rs
Line 59 in 58cc4e1
Looks to me like there are basically 2 things to do:
OptimizerRule::rewriteAPI (as in implement rewrite for FilterNullJoinKeys #10166)expr.clone()calls heredatafusion/datafusion/optimizer/src/single_distinct_to_groupby.rs
Line 131 in 58cc4e1
Once you do 1. 2 should be pretty simple (maybe even just remove the
.clone()appletreeisyellow commentedon May 13, 2024
@alamb Thank you for the guidance!