Repository navigation
Statistics::total_byte_size does not account for projection in FileScanConfig::with_projection #14936
Description
Describe the bug
As @blaginin found in #14685, the statistics when a File is projected (aka only a subset of the columns are present) is incorrect
Specifically, the projected statistics have the same total_byte_size as the input. However, given only a subset of columns are selected this will mean that the total_byte_size should actually be lower
To Reproduce
See tests referenced in #14685
Expected behavior
total_byte_size should take into account the subset of columns
Additional context
No response
Activity
blaginin commentedon Mar 24, 2025
Should be easier to fix after #15352
use ProjectionExprs:project_statistics in FileScanConfig
use ProjectionExprs:project_statistics in FileScanConfig
use ProjectionExprs:project_statistics in FileScanConfig
use ProjectionExprs:project_statistics in FileScanConfig (#19094)
adriangb commentedon Dec 8, 2025
Hi folks although #19094 technically closed this (the value is correct) we can get better estimates with something like #19113. Would anyone that was interested in this issue like to review that work?
Track column sizes in Statistics; propagate through projections (#19113)