Repository navigation
Library Guide: Building LogicalPlans #7306
Description
Is your feature request related to a problem or challenge?
Part of #7014
If we want to have DataFusion used as the core of many new systems, we need it to be as easy as possible for someone to get their idea working on top of DataFusion.
Thanks to @tshauck we now have a basic Library Users Guide ❤️ and this ticket describes expanding it out
Describe the solution you'd like
Fill in the content of https://arrow.apache.org/datafusion/library-user-guide/building-logical-plans.html
Example Outline
- Building LogicalPlans directly
- Building LogicalPlans using
LogicalPlanBuilder - Translating LogicalPlan --> ExecutionPlan via physical planner
The examples directory holds a bunch more of examples: https://github.com/apache/arrow-datafusion/tree/main/datafusion-examples
Describe alternatives you've considered
No response
Additional context
No response
Activity
edmondop commentedon May 24, 2024
@alamb I checked the previous @andygrove PR and it seems like we have a good setup. I read the docs, and noticed they lack a paragraph on how to translate the LogicalPlan into Physical Plan .
What do you think we should do? I can add that paragraph, but maybe we also want to link the examples? What else do you think it's needed to close this
alamb commentedon May 25, 2024
Those both sound like excellent ideas to close off this issue. Thank you @edmondop
edmondop commentedon May 25, 2024
I noticed that the original work from @andygrove referred examples in the docs and these seems to be pretty "unique" in the sense that no other doc page seems to take this approach , see https://github.com/apache/datafusion/tree/main/docs/src
What do you think about moving those into examples and linking them from the docs? Do we have a preprocessor macro to includes the Rust code in a markdown upon compilation, or maybe to ensure that the embedded snippets compile and execute correctly ?
alamb commentedon Jun 3, 2024
I think @andygrove started on a pre-processor that extracted the markdown and tested it here #7956
Maybe we can revive that idea
edmondop commentedon Jun 3, 2024
I have created #10768 which I think would be a replacement of the Python scripts with something maintained somewhere else
Add new user doc to translate logical plan to physical plan
jc4x4 commentedon Aug 16, 2024
I've added a first draft to the doc. Let me know if you would look for a more sophisticated example.
Add new user doc to translate logical plan to physical plan (#12026)