Skip to content

Deduplicate Concat string and binary builders in datafusion-functions #26304

Closed

Description

@andygrove

Is your feature request related to a problem or challenge?

datafusion-functions has two near-identical sets of Concat builders, about 230 lines each:

Both contain an offset-based builder and a view builder. The with_capacity, append_offset and finish bodies and the 7-arm ColumnarValueRef match in write differ only in the target array type, a UTF-8 check on scalars, and .as_bytes() on string values. The view builders' append_offset (make_view, the 12-byte inline threshold and the u32 offset overflow check) is identical in both, so any fix to the view layout logic has to be made twice.

Describe the solution you'd like

  • Introduce one byte-oriented builder parameterised over the output array type (a small trait providing the data type, the final array construction, and a scalar-validation hook).
  • Add a helper on ColumnarValueRef such as value_bytes(i) -> Option<&[u8]> so the 7-arm match collapses to one call.
  • Share the view-append logic between ConcatBinaryViewBuilder and ConcatStringViewBuilder, keeping only the type-specific finish() and its SAFETY comments separate.

Describe alternatives you've considered

Share only the view-builder append_offset, which is the most delicate duplicated piece.

Additional context

Line references are pinned to commit 0d79252.

Activity

  1. added
    enhancementNew feature or request
    help wantedExtra attention is needed
    functionsChanges to functions implementation
    tech debtCode complexity, duplication or structure that makes the codebase harder to change
    on Oct 14, 2026
  2. self-assigned this
    on Oct 14, 2026
  3. added a commit that references this issue on Oct 19, 2026
    a99fb4a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestfunctionsChanges to functions implementationhelp wantedExtra attention is neededtech debtCode complexity, duplication or structure that makes the codebase harder to change

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions