Skip to content

Enable parquet page level skipping (page index pruning) by default #4085

Closed

Description

@alamb

In #3834 and #847 @Ted-Jiang added support for skipping rows in the parquet scan via the RowIndex mechanism 🎉

This feature is currently enabled via a feature flag, which is disabled by default.

This ticket tracks enabling this feature by default.

Currently known items are:

Related to #3463

Activity

  1. self-assigned this
    on Mar 13, 2023
  2. alamb commented on Mar 27, 2023

    @alamb
    ContributorAuthor

    BTW update here is the latest benchmark runs I did -- #5099 (comment) -- show some small performance regressions we need to fix in order to turn this on by default

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions