Repository navigation
Blog post with DataFusion Jun - Sep 2023 #6780
Description
Is your feature request related to a problem or challenge?
We have had good luck writing up quarterly updates for DataFusion, most recently:
https://arrow.apache.org/blog/2023/06/24/datafusion-25.0.0/
(see #5812)
Describe the solution you'd like
It would be great to write another about what has happened in the last few months of DataFusion
Things I expect will be good to highlight (🤞 ):
- Improved Struct/array support (@izveigor ❤️ )
- better group by performance with many distinct groups
- better insert performance
Others?
Describe alternatives you've considered
No response
Additional context
No response
Reacted by Igor Izvekov, Ken, Wang and Matthew Powers
Activity
alamb commentedon Jul 17, 2023
Ideas of Major items to include in this post
Aggregator, grouping, aggregation #4973Dandandan commentedon Jul 17, 2023
Improved join performance would maybe be another thing to highlight. Maybe we show a benchmark with improvements (TCP-H, ClickBench, ...) from version 25 -> 28.
alamb commentedon Aug 17, 2023
There has been major work on
INSERTandCOPYas well, thanks to @devinjdangelo : #6569alamb commentedon Sep 15, 2023
Also #7400 spilling group by from @kazuyukitanimura
alamb commentedon Sep 18, 2023
Another topic: the new library user guide: https://arrow.apache.org/datafusion/library-user-guide/index.html
alamb commentedon Oct 14, 2023
FYI this is very much on my list, but I need to focus on the SIGMOD paper for a while. If someone else has the time and inclination to start a PR I would be most appreciative
alamb commentedon Nov 6, 2023
Realistically I am very tied up with #6782 and so won't have time to work on a blog post until after that is submitted (end of Nov). If someone else has time to work on this it would be very much apprecaited
alamb commentedon Jan 1, 2024
This is going to have to be more like a 2023 retrospective 🤔
alamb commentedon Jan 4, 2024
I am starting to draft this now
alamb commentedon Jan 7, 2024
Here is a PR with a draft (still needs more work): apache/arrow-site#457
[Website]: DataFusion 26-34 blog post (#457)
alamb commentedon Jan 19, 2024
The blog post is now published! https://arrow.apache.org/blog/2024/01/19/datafusion-34.0.0/
alamb commentedon Mar 13, 2024
Let's capture other items to highlight here #9602