Closed
Description
Is your feature request related to a problem or challenge?
Part of #10922
We are adding APIs to efficiently convert the data stored in Parquet's "PageIndex" into ArrayRef
s -- which will make it significantly easier to use this information for pruning and other tasks.
Describe the solution you'd like
Add support to StatisticsConverter::min_page_statistics
and StatisticsConverter::max_page_statistics
for the types above
datafusion/datafusion/core/src/datasource/physical_plan/parquet/statistics.rs
Lines 637 to 656 in a923c65
Describe alternatives you've considered
You can follow the model from @Weijun-H in #10931
- Update the test for the listed data types to be
Check::Both
, following the model oftest_int64
datafusion/datafusion/core/tests/parquet/arrow_statistics.rs
Lines 506 to 529 in a923c65
- Add any required implementation in
get_datapage_statistics
: (follow the model of the row counts, )
Typically the change to the test looks like
- check: Check::RowGroup,
+ check: Check::Both,
Additional context
No response