Repository navigation
unexpected output for concat for arrays #18020
Description
Describe the bug
concat builtin in function should concat arrays like array_concat, currently the output is wrong
> select concat(make_array(1, 2, 3), make_array(4, 5));
+------------------------------------------------------------------------------+
| concat(make_array(Int64(1),Int64(2),Int64(3)),make_array(Int64(4),Int64(5))) |
+------------------------------------------------------------------------------+
| [1, 2, 3][4, 5] |
+------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.012 seconds.
> select array_concat(make_array(1, 2, 3), make_array(4, 5));
+------------------------------------------------------------------------------------+
| array_concat(make_array(Int64(1),Int64(2),Int64(3)),make_array(Int64(4),Int64(5))) |
+------------------------------------------------------------------------------------+
| [1, 2, 3, 4, 5] |
+------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.014 seconds.
concat for array should return same as array_concat
To Reproduce
No response
Expected behavior
No response
Additional context
No response
Activity
petern48 commentedon Oct 12, 2025
take
petern48 commentedon Oct 12, 2025
I see that the string concat function is being called instead of function-nested array_concat one. Adding a
concatalias toarray_concattechnically solves the problem. Is that what we want? Seems reasonable to me, but I just didn't see any signs that we want / expected this to be already be an alias.comphead commentedon Oct 12, 2025
Thanks @petern48 for looking into this issue. I'm not sure how would be easy to make an alias depending on the signature. in the end of the day we need to have
concatwith arrays working properly and this is current behavior in DuckDB.Naive solution is to replace
concatfunction toarray_concatin the planner depending on the inputpetern48 commentedon Oct 15, 2025
I see, so it sounds like we want to use the current
concatimplementation only for strings and then usearray_concatimplementation for the other data types?I tried finding an approach where we didn't have to replace the function call in the planner. I tried adding
concatas an alias toarray_concat, hoping thatconcat.rswould be called for strings andarray_concatwould work for everything else, butarray_concatseemed to just override theconcat.rsudf entirely (saying there's no valid kernel for Utf8 args), so the existing concat tests failed.Anyways, given my current bandwidth, I'd rather leave this for someone else to debug / propose a solution. Could you unassign me @comphead? Doesn't seem like I can do it myself.
divyanshmrawal commentedon Oct 16, 2025
We have to use array_concat for concatenating arrays because concat is used for strings, and array_concat is used for concatenating arrays.
EeshanBembi commentedon Oct 16, 2025
I'd like to take a shot at this issue. After examining the codebase, I think there's a clean solution that doesn't require planner changes or function aliases.
The key insight is that we can modify the existing concat function to detect when array types are passed and delegate to the array_concat logic internally, while preserving all existing string concatenation behavior.
Proposed approach:
FixedSizeList) in addition to string types
type, delegate to array_concat_inner
The implementation would be contained within
/datafusion/functions/src/string/concat.rs with minimal changes to the existing
logic flow.
@comphead Would this approach work?
comphead commentedon Oct 16, 2025
Thanks @EeshanBembi it might actually work, would be nice if you create a draft PR and explore how that would work.
comphead commentedon Oct 17, 2025
I played with replacement in the planner, but somehow I dont feel like this is a good idea.
@EeshanBembi I have attached a PR just for reference, please feel free to take tests from the PR, tests for both DF
concatand Spark should pass(they are slightly different)feat: Add array concatenation support to concat function
EeshanBembi commentedon Oct 17, 2025
@comphead Thanks for the reference PR! After reviewing #18105, I believe the
function-level approach is better:
across the planner
planning
The implementation is complete with full test coverage and 100% backward
compatibility. Creating PR now with the working solution.
4 remaining items
feat: delegate array concatenation from concat to array_concat logic
feat: delegate array concatenation from concat to array_concat logic
feat: add minimal array concatenation support to concat function
feat: add array concatenation support to concat function
feat: delegate array concatenation from concat to array_concat logic
ruthvikgowda814-bot commentedon Nov 30, 2025
Example
select concat(make_array(1, 2, 3), make_array(4, 5));
output
[1, 2, 3][4, 5]
correct behavior
select array_concat(make_array(1, 2, 3), make_array(4, 5));
output
[1, 2, 3, 4, 5]
EeshanBembi commentedon Nov 30, 2025
Hey @ruthvikgowda814-bot
This is fixed in PR #18137 (branch EeshanBembi:feature/concat-array-support)
Just tested and concat now returns [1, 2, 3, 4, 5] correctly. Did you test using this branch?
KARTIK64-rgb commentedon Feb 21, 2026
Hi!
I’d really like to work on this issue.
From what I understand, the problem is that when concat() receives array arguments, it currently concatenates their string representations instead of behaving like array_concat() and returning a properly merged flat array.
I’m planning to review the concat function’s dispatch logic and add a check to ensure array-type inputs are handled the right way.
Could I please be assigned this issue? Thanks!
Jefffrey commentedon Feb 23, 2026
Perhaps can check with @EeshanBembi if they still plan to continue with the current PR for this issue
I would suggest taking a look at it as there are things we encountered that make this non-trivial to fix
KARTIK64-rgb commentedon Feb 23, 2026
sure
1fanwang commentedon Sep 24, 2026
take
1fanwang commentedon Sep 24, 2026
untake