Support optional filter in SortMergeJoin #2628

Activity

1. yjshen

   added

   enhancementNew feature or requestNew feature or request

   on May 27, 2022
2. korowa commented on May 31, 2022

   

   korowa

   on May 31, 2022

   Last edited by korowa

   Contributor

   More actions

   I've tried to do some POC with constructing intermediate batch and applying filter to it while freeze_buffered_join_streamed -- it seems to be the only place where filtering required, and noticed (please correct me, if I'm mistaken), that due to freeze_* functions logic, output ordering can be broken in case of outer joins -- while freezing, joined and non-joined records from outer table append as separate batches, and after that no merges / resorts happen -- just batch concatenation.

   I suppose output ordering to be quite important for planning (i.e. if we had merge/stream/sorted aggregate operator it could be planned over merge join output instead of default HashAggregate), so I wonder - shouldn't this be fixed prior to MJ filter? I guess this fix could significantly change MJ logic in places where filtering required 🤔 -- if so, I can file another issue and dive deeper into MJExec logic.

   @yjshen , @richox what do you think of it?
3. Dandandan commented on Dec 5, 2024

   

   Dandandan

   on Dec 5, 2024

   Contributor

   More actions

   AFAIK closed by #9080
4. Dandandan

   closed this as completedon Dec 5, 2024