Repository navigation
Implement flatten function #6995
Description
Is your feature request related to a problem or challenge?
Summary
| Characteristic | Description |
|---|---|
| Function name: | flatten |
| Aliases: | - |
| Original function?: | No |
| Function Description: | Azure DataBricks: Returns an array of the elements in the union of array1 and array2 without duplicates. Spark SQL: Creates a single array from an array of arrays column. ClickHouse: Converts an array of arrays to a flat array. Function: Applies to any depth of nested arrays. Does not change arrays that are already flat. The flattened array contains all the elements from all source arrays. DuckDB: Concatenate a list of lists into a single list. This only flattens one level of the list (see examples). |
| Sources: | Concept Azure DuckDB ClickHouse Spark |
Examples:
D select flatten([[1, 2, 3], [4, 5, 6]]);
┌──────────────────────────────────────────────────────────────────────────────┐
│ flatten(main.list_value(main.list_value(1, 2, 3), main.list_value(4, 5, 6))) │
│ int32[] │
├──────────────────────────────────────────────────────────────────────────────┤
│ [1, 2, 3, 4, 5, 6] │
└──────────────────────────────────────────────────────────────────────────────┘
D select flatten([[[1, 1], [2, 2], [3, 3]], [[4, 4], [5, 5], [6, 6]]]);
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ flatten(main.list_value(main.list_value(main.list_value(1, 1), main.list_value(2, 2), main.list_value(3, 3)), main.list_value(main.list_value(4, 4), main.li… │
│ int32[][] │
├────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ [[1, 1], [2, 2], [3, 3], [4, 4], [5, 5], [6, 6]] │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
Activity
jayzhan211 commentedon Aug 6, 2023
Let me take a look on this issue.
izveigor commentedon Aug 6, 2023
That sounds good, @jayzhan211! I only want to warn you that
DuckDBversion offlattenfunction is different from the rest (decreases only 1 level than the rest to 1 dimensional array). I think the last option is much better (ClickHouse, Spark and Azure DataBricks).jayzhan211 commentedon Aug 7, 2023
For the latter version, it seems the function is implemented in #6796. I will continue on this after that PR merged.