Repository navigation
[C++] Support string collation for sorting #27875
Description
Currently the C++ library orders strings lexicographically as bytestrings. We should implement the capability to change string sorting behavior based on locale settings for string collation.
Reporter: Ian Cook / @ianmcook
Note: This issue was originally created as ARROW-12046. Please see the migration documentation for further details.
Activity
asfimport commentedon Mar 22, 2021
Ian Cook / @ianmcook:
cc @pitrou
asfimport commentedon Mar 22, 2021
Antoine Pitrou / @pitrou:
The current locale is a global process setting, so Arrow shouldn't change it. What we may add is a "compare strings using the current locale" option, though.
asfimport commentedon Mar 22, 2021
Antoine Pitrou / @pitrou:
Note this would only work probably if a UTF-8 locale is selected.
asfimport commentedon Apr 21, 2021
Antoine Pitrou / @pitrou:
DuckDB has built a stripped down version of ICU, which might be useful to us:
https://github.com/duckdb/duckdb/tree/master/extension/icu
https://github.com/Mytherin/minimal-icu-collation
asfimport commentedon Jun 28, 2021
Antoine Pitrou / @pitrou:
Also see jecolon/ziglyph#3
asfimport commentedon Oct 12, 2021
Benson Muite / @bkmgit:
Compression will be helpful. One way of sorting Mandarin is by stroke count, for which there is a table for unicode However, this is not the only way so allowing developers freedom to create appropriate comparison functions is useful. Minimal-icu-collation also seems helpful, but is not vectorized. Javascript has a specification for comparison functions - while there will be a loss of efficiency, something like this seems most appropriate. An example of using this can be found in this DataTables blog. QString has a locale aware comparison but licensing only allows incorporation of Apache code into GPL or LGPL code, and not the other way round.
asfimport commentedon Dec 14, 2021
Benson Muite / @bkmgit:
A related issue is the between kernel for string comparisons https://issues.apache.org/jira/browse/ARROW-9843
asfimport commentedon Dec 28, 2021
Benson Muite / @bkmgit:
Sort Keys can be used:
https://arrow.apache.org/docs/cpp/api/compute.html#_CPPv4N5arrow7compute11SortOptionsE
Is this a reasonable approach to build on? This is used for example in R
https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/xtfrm
github-actions commentedon Jan 16, 2026
This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.