Repository navigation
Run / Test all examples in Documentation #14435
Description
Is your feature request related to a problem or challenge?
The https://datafusion.apache.org/library-user-guide/index.html has some great examples ❤
However, not all of these examples are actually tested during CI which results in issues like this one found (and fixed!) by @nuno-faria
It is possible to prevent this type of problem by testing the examples automatically, following the instructions here
datafusion/datafusion/core/src/lib.rs
Lines 819 to 844 in 67bc04c
Describe the solution you'd like
I would like to test all examples in the entire User Guide and Library User Guide.
Here are the files in the docs/source/library-user-guide directory. The ones that are not checked do not have their examples checked yet
-
docs/source/library-user-guide/adding-udfs.md -
docs/source/library-user-guide/api-health.md -
docs/source/library-user-guide/building-logical-plans.md -
docs/source/library-user-guide/catalogs.md -
docs/source/library-user-guide/custom-table-providers.md -
docs/source/library-user-guide/extending-operators.md -
docs/source/library-user-guide/extensions.md -
docs/source/library-user-guide/index.md -
docs/source/library-user-guide/profiling.md -
docs/source/library-user-guide/query-optimizer.md -
docs/source/library-user-guide/using-the-dataframe-api.md -
docs/source/library-user-guide/using-the-sql-api.md -
docs/source/library-user-guide/working-with-exprs.md
Describe alternatives you've considered
To test the examples on a particular page, such as adding-udfs.md
Step 1: Add entry to lib.rs
andrewlamb@Andrews-MacBook-Pro-2:~/Software/datafusion$ git diff
diff --git a/datafusion/core/src/lib.rs b/datafusion/core/src/lib.rs
index 780b22983..509ef601c 100644
--- a/datafusion/core/src/lib.rs
+++ b/datafusion/core/src/lib.rs
@@ -873,6 +873,12 @@ doc_comment::doctest!(
user_guide_expressions
);
+#[cfg(doctest)]
+doc_comment::doctest!(
+ "../../../docs/source/library-user-guide/adding-udfs.md",
+ library_user_guide_adding_udfs
+);
+
#[cfg(doctest)]
doc_comment::doctest!(
"../../../docs/source/library-user-guide/using-the-sql-api.md",Step 2: Run tests:
cargo test --doc -- library_user_guide_adding_udfs
Step 3: Fix issues / run again
(iterate and repeat until test passes)
Additional context
No response
Activity
[-]Run / Test all examples in Documenation[/-][+]Run / Test all examples in Documentation[/+]alamb commentedon Feb 3, 2025
I think this would be a good first issue for people to work on as it is self contained, well described, doesn't require deep internals experience, and would give you exposure to the API
Chen-Yuan-Lai commentedon Feb 3, 2025
@alamb I found that some tests failed because multiple code blocks shared some imports. For example, should we repeat the necessary imports in the second block?
ugoa commentedon Feb 3, 2025
Hey I would like to have a try, I am quite interested in the datafusion and daft project and have been exploring both recently
alamb commentedon Feb 3, 2025
Yes I think that is what is needed for now.
If that makes the documentation too messy maybe we can figure out how to do the rustdoc style of hiding lines that start with
#Thanks @Chen-Yuan-Lai
alamb commentedon Feb 3, 2025
Thank you @ugoa -- that is great!
I recommend you pick of of the files above and then work through the errors to get all the tests passing
ugoa commentedon Feb 4, 2025
Hey @alamb @Chen-Yuan-Lai folks, here is my attempt to fix doctest for file
docs/source/library-user-guide/adding-udfs.md, the below test command is able to pass successfully:cargo test --doc -- library_user_guide_adding_udfsHowever, I do need advice on how to improve the issue pointed out by @Chen-Yuan-Lai. For this MR I take the (#) approach as suggested, but the doc becomes quite long, and the build process is also required to change because the
sphinxdoesn't support therustdocformat (I think?).Anyway looking forward to your feedback, cheers!