Plain SELECT of a Parquet float column returns wrong values: NaN rows are replaced by the statistics constant when min == max #26490

Describe the bug

Parquet writers leave NaN out of floating-point min/max statistics (the spec requires it). So a column chunk holding
one distinct finite value plus some NaNs has statistics min == max == x and null_count == 0, although its values are
not constant. DataFusion treats such a column as constant and substitutes the statistic for the data. A plain
SELECT * FROM 'file.parquet' then returns x for every NaN row. There is no error, and NaNs silently disappear.
pyarrow, DuckDB and Polars read the same file correctly.

This is not limited to pruning: datafusion.execution.parquet.pruning = false and enable_page_index = false do not
change the result. datafusion.execution.collect_statistics = false does. A COPY … TO from such a file writes the
wrong values into the new files.

The shortcut is constant_columns_from_stats / constant_value_from_stats in
datafusion/datasource-parquet/src/opener/mod.rs (lines 1968 and 1999, called at line 891): min == max && !min.is_null() && null_count == Exact(0). For Float16/32/64, Parquet min/max cannot prove the column is constant, because NaN is excluded from them.
So floating-point columns should not be treated as constant from these statistics (unless a NaN count of 0 is known).

Draft PR #24804 (for #15812, pruning with NaN) would disable floating-point min/max in a way that also covers this
("constant-column substitution"), but it is a draft and has no issue for this user-visible symptom. I'm filing it so
the wrong-result case is tracked and tested.

To Reproduce

import pyarrow as pa, pyarrow.parquet as pq
pq.write_table(pa.table({'c0': pa.array([float('nan'), -8.6e-27])}), 'nan.parquet')
pq.write_table(pa.table({'c0': pa.array([1.5, float('nan')], pa.float32())}), 'nan_f32.parquet')
pq.write_table(pa.table({'c0': pa.array([float('nan'), -8.6e-27])}), 'nan_nostats.parquet', write_statistics=False)

$ datafusion-cli -q -c "SELECT * FROM 'nan.parquet';" -c "SELECT count(*) FILTER (WHERE isnan(c0)) AS nans FROM 'nan.parquet';"
+----------+
| c0       |
+----------+
| -8.6e-27 |
| -8.6e-27 |      <-- this row is NaN in the file
+----------+
+------+
| nans |
+------+
| 0    |
+------+

$ datafusion-cli -q -c "SELECT * FROM 'nan_f32.parquet';"
| 1.5 |
| 1.5 |            <-- NaN in the file

$ datafusion-cli -q -c "SELECT * FROM 'nan_nostats.parquet';"
| NaN      |
| -8.6e-27 |       <-- correct without statistics

$ datafusion-cli -q -c "SET datafusion.execution.collect_statistics = false;" -c "SELECT * FROM 'nan.parquet';"
| NaN      |
| -8.6e-27 |       <-- correct

The same happens for files written by DataFusion's own COPY. [NaN, 2.0, 3.0] (min ≠ max) and [NaN, NaN] (no min/max
written) are read correctly.

Both column orders are affected. The pyarrow files above have TYPE_DEFINED_ORDER and no nan_count. DataFusion/arrow-rs
60 writes IEEE_754_TOTAL_ORDER with nan_count, but still leaves NaN out of min/max: COPY (VALUES ('NaN'::double), (5.0))
gives min = max = 5.0, null_count = 0, nan_count = 1, and that file is read back as 5.0, 5.0. So DataFusion ignores a
nan_count that the file does carry.

Expected behavior

SELECT * FROM 'nan.parquet' returns NaN, -8.6e-27, as pyarrow, DuckDB and Polars do. Float statistics with min == max
should not be used to replace the column by a constant, because NaN values are not reflected in them.

Additional context

• DataFusion main 290ad15 (2026-10-18), datafusion-cli 55.1.0 built from source; also 91cd7ff (2026-10-15). pyarrow 26.0.0.
• Shapes like "one default value plus missing values as NaN" are common in sensor and ML feature data.
• Related: Pruning of floating point Parquet columns is incorrect when NaN is present #15812 and draft PR fix: prevent incorrect Parquet pruning with NaNs #24804.

Willingness to contribute

None