Repository navigation
Potential performance regression with comparisions to scalar values #14291
Description
Describe the bug
There is concern changes in this PR will cause regression (as it may convert some numbers to Decimal128 rather than more efficient Integer)
However, it is not clear (to me at least) exactly the implications of the change
@berkaysynnada has found a potential performance regression, as described here: #14223 (comment)
cc @nuno-faria @jonahgao and @berkaysynnada and @ozankabak
To Reproduce
No response
Expected behavior
I want to make sure that this issue is resolved before we release datafusion 45.0.0, so filing this ticket so we don't forget
Examples of ways to resolve the ticket:
- Decide the change was fine
- Revert the changes
- Others
Additional context
No response
Activity
alamb commentedon Jan 25, 2025
I ran benchmarks with / without this change and did not see any noticable performance difference. See details here
I also created a PR with some tests that show the practical impact of this PR on comparisons (I think it is minimal). See
So in other words I would be inclined to close this ticket without further changes given what I currently know
berkaysynnada commentedon Jan 25, 2025
https://github.com/synnada-ai/datafusion-upstream/tree/benchmark-cast
The last commit shows the simplest case that is regressed