Repository navigation
Implement array_intersect function #6978
Description
Is your feature request related to a problem or challenge?
Summary
| Characteristic | Description |
|---|---|
| Function name: | array_intersect |
| Aliases: | list_intersect, && |
| Original function?: | No |
| Function Description: | Azure DataBricks: Returns an array of the elements in the intersection of array1 and array2. Clickhouse: Takes multiple arrays, returns an array with elements that are present in all source arrays. DuckDB: Returns a list of all the elements that exist in both l1 and l2, without duplicates. |
| Sources: | Concept Azure ClickHouse DuckDB |
Examples:
select array_intersect([1, 2, 3, 4], [5, 6, 3, 4]);
----
[3, 4]
select array_intersect([1, 2, 3, 4], [5, 6, 7, 8]);
----
[]
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
Activity
Veeupup commentedon Nov 6, 2023
Hi, I'm new to datafusion, could I have a try with this one?
alamb commentedon Nov 7, 2023
Thanks @Veeupup -- just be aware these arrays / functions are quite tricky (you might ask @edmondop who is working on array union in #7897)
Another place that could use help (and might be a easier introduction to DataFusion) could be to help @jayzhan211 help clean up the existing array functions (e.g #8071) -- you would get to see how the existing ones work before having to implement your own
edmondop commentedon Nov 7, 2023
@Veeupup I confirm, this is a change that probably require some tens of hours between studying, trial and errors, interacting with the datafusion community. You probably want to start with something simpler
jayzhan211 commentedon Nov 7, 2023
If starting from the basic scalar, non-column, each implementation for each type, it is much easier, but the general type approach is really non-trivial
jayzhan211 commentedon Nov 7, 2023
I recommend starting from array_append/prepend. #7988
But if you want to challenge yourself, array_intersect is also a choice, I think no one is working on it yet.
Veeupup commentedon Nov 7, 2023
Thank you all! ^_^
I have been working for array_intersect for maybe 1 hour... And I'm happy to start from array_append/prepend.
And maybe I'll come back later when I finish array_append/prepend!