Skip to content

Generic Rows-backed GroupColumn so nested GROUP BY keys stay on the column-wise fast path #23404

Open

Description

@zhuqi-lucas

Part of #22715.

Problem

GroupValuesColumn (the column-wise multi-column group-by path) is all-or-nothing: if a single group-by column has no type-specialized GroupColumn, supported_schema returns false and the entire key falls back to the row-wise GroupValuesRows — even when every other column would have qualified for the column-wise fast path. Nested types (Struct, List, FixedSizeList, ...) currently trigger this, so a mixed key like GROUP BY int_col, struct_col loses the fast path (and its lower memory footprint) entirely because of that one column.

Proposal

(Suggested by @alamb in #23128 (comment).)

Add a single generic GroupColumn backed by arrow's row format (a one-field RowConverter) as a fallback. Then GroupValuesColumn stays usable for any row-encodable type, and row-encoding is confined to just the columns that lack a native builder — instead of dropping the whole key onto GroupValuesRows. Hashing is unchanged (done on the raw input columns, which already supports nested types).

Scope (first step)

The generic RowsGroupColumn framework + routing for the nested types RowConverter can encode (Struct / List / LargeList / FixedSizeList, recursively). A quick local prototype suggests a mixed native + nested key uses roughly half the memory of the all-rows fallback, with grouping identical to the GroupValuesRows reference.

Follow-ups

  • Cover types RowConverter cannot encode (e.g. Map, which needs key-order normalization) so supported_schema can always be true.
  • With full coverage, retire GroupValuesRows entirely.
  • Batch-scoped cache for the scalar equal_to / append_val path (needs a GroupColumn batch hook).

Activity

  1. self-assigned this
    on Jul 9, 2026
  2. alamb commented on Jul 9, 2026

    @alamb
    Contributor

    With full coverage, retire GroupValuesRows entirely.

    I think this would be really nice -- to be able to remove the whole thing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions