Repository navigation
Library Guide: Add SQL level user guide: #7302
Description
Is your feature request related to a problem or challenge?
Part of #7014
If we want to have DataFusion used as the core of many new systems, we need it to be as easy as possible for someone to get their idea working on top of DataFusion.
Thanks to @tshauck we now have a basic Library Users Guide ❤️ and this ticket describes expanding it out
Describe the solution you'd like
Fill in the content of https://arrow.apache.org/datafusion/library-user-guide/using-the-sql-api.html
Example outline
- Generating LogicalPlans from
sql - Example / guide of how to extend SQL (both links to user defined functions and custom nodes, but also maybe some links to the datafusion sql variant parser
- Example of how to run SQL read only and how to run it with catalog modifications (Misleading/invalid documentation on SessionContext::sql and SessionConfig::create_logical_plan #7272)
The examples directory holds a bunch of examples: https://github.com/apache/arrow-datafusion/tree/main/datafusion-examples
Describe alternatives you've considered
No response
Additional context
No response
Activity
qstommyshu commentedon Mar 24, 2025
Hi, I can try to add some documentation for SQL level user guide as I'm exploring the project.
alamb commentedon Mar 24, 2025
Thaks @qstommyshu
Something that that would be super helpful would be to move some of the basic examples from the examples directory into the documentation. Previously the code in examples was not tested but it is now.
For example, moving this example might be a great place to start
And remove that example function and add it as examples in https://datafusion.apache.org/library-user-guide/using-the-sql-api.html
qstommyshu commentedon Mar 24, 2025
Sounds good, I'll start with the example then!
qstommyshu commentedon Mar 26, 2025
Hi @alamb
I've read through some examples in main/datafusion-examples/examples/, they are really helpful for understanding how to use datafusion.
And I've checked and tested examples in library user guide, the examples are correct and runnable (I copied the examples from doc and pasted it to
sql_query.rs, everything runs flawlessly).I made some code refactoring to the examples in /datafusion-examples/examples/sql_query.rs on local just to make it easier for a beginner to learn about datafusion. Maybe I can create a PR from here?
That being said, I think we probably don't need to remove example functions in the library user guide.
I'll try to fill out the documentation content as I'm exploring the project.