Repository navigation
Eliminate more outer joins by supporting more expressions #13232
Description
Is your feature request related to a problem or challenge?
This PR by @JasonLi-cn tried to support more pushdown of filters #13184 , however the implementation wasn't 100% right as it pushed down more predicates, but didn't change the join type (left to inner).
Describe the solution you'd like
Add support for a wider range of expressions in EliminateOuterJoin, notably in extract_non_nullable_columns.
We lack support of more complex nested expressions, quite some binary operators, UDFs etc. such as:
abs(x) > 1a + a>c + b
We should aim to support any operator that returns null when the input value is null (most binary / unary operators).
Describe alternatives you've considered
No response
Additional context
No response
Reacted by JasonLi-cn, kamille and Jonathan Chen
Activity
[-]Eliminate more outer joins by supporting more expressions, like UDFs[/-][+]Eliminate more outer joins by supporting more expressions[/+]Rachelint commentedon Nov 4, 2024
take
JasonLi-cn commentedon Nov 4, 2024
I have implemented a preliminary version and am happy to solve this Issue together. 😄
pr: #13249
Rachelint commentedon Nov 4, 2024
😆 OK, I am learning it!
EliminateOuterJoin with Like, IsTrue, IsFalse, IsNotUnknown (#21549)
EliminateOuterJoin with Like, IsTrue, IsFalse, IsNotUnknown (apache#2…
EliminateOuterJoin with Like, IsTrue, IsFalse, IsNotUnknown (apache#2…
neilconway commentedon May 26, 2026
#22444 should have improved this situation somewhat. We still don't consider UDFs to ever be null-rejecting, which is a fairly big hole.
neilconway commentedon May 26, 2026
#22534 is another incremental improvement here.
feat: add strictness metadata for scalar UDF null propagation and use…
DKER2 commentedon Oct 11, 2026
#26196 sweep through functions and add is_strict metadata to help outer joins rewrite decide better
Mark null-propagating regex and date/time functions as strict
Mark null-propagating core, hash, array and Spark functions as strict