Hide boilerplate in documentation examples

@ugoa

Is your feature request related to a problem or challenge?

Part of #7013

@ugoa has updated our examples in the following PR to all compile and be tested

Test all examples from library-user-guide & user-guide docs #14544,

One major problem I have with using # to hide setup code, is that the sphinx won't respect the leading # and still renders the whole block, which leading huge chunk of ugly code blocks.

Describe the solution you'd like

I would like to be able to hide the setup code for the examples, following the rustdoc convention of ignoring lines that begin with #

For example, in our docs here:

datafusion/datafusion/core/src/lib.rs

Lines 59 to 94 in 91c0975

    
           //! ```rust 
        
           //! # use datafusion::prelude::*; 
        
           //! # use datafusion::error::Result; 
        
           //! # use datafusion::functions_aggregate::expr_fn::min; 
        
           //! # use datafusion::arrow::record_batch::RecordBatch; 
        
           //! 
        
           //! # #[tokio::main] 
        
           //! # async fn main() -> Result<()> { 
        
           //! let ctx = SessionContext::new(); 
        
           //! 
        
           //! // create the dataframe 
        
           //! let df = ctx.read_csv("tests/data/example.csv", CsvReadOptions::new()).await?; 
        
           //! 
        
           //! // create a plan 
        
           //! let df = df.filter(col("a").lt_eq(col("b")))? 
        
           //!            .aggregate(vec![col("a")], vec![min(col("b"))])? 
        
           //!            .limit(0, Some(100))?; 
        
           //! 
        
           //! // execute the plan 
        
           //! let results: Vec<RecordBatch> = df.collect().await?; 
        
           //! 
        
           //! // format the results 
        
           //! let pretty_results = arrow::util::pretty::pretty_format_batches(&results)? 
        
           //!    .to_string(); 
        
           //! 
        
           //! let expected = vec![ 
        
           //!     "+---+----------------+", 
        
           //!     "| a | min(?table?.b) |", 
        
           //!     "+---+----------------+", 
        
           //!     "| 1 | 2              |", 
        
           //!     "+---+----------------+" 
        
           //! ]; 
        
           //! 
        
           //! assert_eq!(pretty_results.trim().lines().collect::<Vec<_>>(), expected); 
        
           //! # Ok(()) 
        
           //! # }

Lines that start with # are not rendered in the markdown docs on https://docs.rs/datafusion/latest/datafusion/index.html

Describe alternatives you've considered

I wonder if we can find find some way to add a sphinx plugin or something to avoid rendering the blocks of ugly code.

You can build and test the documentation locally by following the instructions here: https://github.com/apache/datafusion/blob/main/docs/README.md

I normally do

cd docs
./build.sh
open build/html/index.html

Additional context

No response

I think this would be a good first issue as it is super helpful, well specified, and doesn't need deep DataFusion internals expertise

@alamb

@alamb assign this to me

@bhavesh26

@alamb assign this to me

Thanks @bhavesh26

Note in the future you can put the comment take and the bot will automatically assign it to you

Please help review workaround solution in 997504f in #14544

@ugoa

Please help review workaround solution in 997504f in #14544

From my perspective the solution your PR is a real fix @ugoa (not just a workaround)

Test all examples from library-user-guide & user-guide docs #14544

Navigation Menu

Repository navigation

Hide boilerplate in documentation examples #14557

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	//! ```rust
	//! # use datafusion::prelude::*;
	//! # use datafusion::error::Result;
	//! # use datafusion::functions_aggregate::expr_fn::min;
	//! # use datafusion::arrow::record_batch::RecordBatch;
	//!
	//! # #[tokio::main]
	//! # async fn main() -> Result<()> {
	//! let ctx = SessionContext::new();
	//!
	//! // create the dataframe
	//! let df = ctx.read_csv("tests/data/example.csv", CsvReadOptions::new()).await?;
	//!
	//! // create a plan
	//! let df = df.filter(col("a").lt_eq(col("b")))?
	//! .aggregate(vec![col("a")], vec![min(col("b"))])?
	//! .limit(0, Some(100))?;
	//!
	//! // execute the plan
	//! let results: Vec<RecordBatch> = df.collect().await?;
	//!
	//! // format the results
	//! let pretty_results = arrow::util::pretty::pretty_format_batches(&results)?
	//! .to_string();
	//!
	//! let expected = vec![
	//! "+---+----------------+",
	//! "\| a \| min(?table?.b) \|",
	//! "+---+----------------+",
	//! "\| 1 \| 2 \|",
	//! "+---+----------------+"
	//! ];
	//!
	//! assert_eq!(pretty_results.trim().lines().collect::<Vec<_>>(), expected);
	//! # Ok(())
	//! # }

Hide boilerplate in documentation examples #14557

Description

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Activity

alamb commented on Feb 8, 2025

bhavesh26 commented on Feb 8, 2025

alamb commented on Feb 8, 2025

ugoa commented on Feb 8, 2025

alamb commented on Feb 10, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions