Support DictionaryString for Regex matching operators

Is your feature request related to a problem or challenge?

While I was working on #12415, I found the DictionaryString can't pass the following case in datafusion/sqllogictest/test_files/string/string_query.slt.part

statement ok
create table test_basic_operator as
select
    arrow_cast(column1, 'Dictionary(Int32, Utf8)') as ascii_1,
    arrow_cast(column2, 'Dictionary(Int32, Utf8)') as ascii_2,
    arrow_cast(column3, 'Dictionary(Int32, Utf8)') as unicode_1,
    arrow_cast(column4, 'Dictionary(Int32, Utf8)') as unicode_2
from test_source;

query BB
SELECT
  ascii_1 ~* '^a.{3}e',
  unicode_1 ~* '^d.*Фу'
FROM test_basic_operator;
----
true false
false false
false true
NULL NULL

I got the error message:

External error: query failed: DataFusion error: Internal error: Data type Dictionary(Int32, Utf8) not supported for binary_string_array_flag_op_scalar operation 'regexp_is_match' on string array.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

Describe the solution you'd like

Support DictionaryString at

datafusion/datafusion/physical-expr/src/expressions/binary.rs

Lines 148 to 158 in 65595cf

    
           macro_rules! binary_string_array_flag_op { 
        
               ($LEFT:expr, $RIGHT:expr, $OP:ident, $NOT:expr, $FLAG:expr) => {{ 
        
                   match $LEFT.data_type() { 
        
                       DataType::Utf8View | DataType::Utf8 => { 
        
                           compute_utf8_flag_op!($LEFT, $RIGHT, $OP, StringArray, $NOT, $FLAG) 
        
                       }, 
        
                       DataType::LargeUtf8 => { 
        
                           compute_utf8_flag_op!($LEFT, $RIGHT, $OP, LargeStringArray, $NOT, $FLAG) 
        
                       }, 
        
                       other => internal_err!( 
        
                           "Data type {:?} not supported for binary_string_array_flag_op operation '{}' on string array",

Describe alternatives you've considered

No response

Additional context

No response

@goldmedal

Thanks @goldmedal -- this is a great find

Related TODO item:

datafusion/datafusion/sqllogictest/test_files/string/string_query.slt.part

Line 645 in c21d025

# TODO: DictionaryString does not support ~* operator. Enable this after fixing the issue
datafusion/datafusion/sqllogictest/test_files/string/large_string.slt

Line 46 in c21d025

# TODO: move it back to `string_query.slt.part` after fixing the issue
datafusion/datafusion/sqllogictest/test_files/string/string.slt

Line 37 in c21d025

# TODO: move it back to `string_query.slt.part` after fixing the issue
datafusion/datafusion/sqllogictest/test_files/string/string_view.slt

Line 40 in c21d025

# TODO: move it back to `string_query.slt.part` after fixing the issue

@goldmedal

I want to take those type issues if you don't mind, @goldmedal and Andrew. It feels like a nice way to get into in the project 😀

take

@blaginin

Thank you @blaginin

BTW here is an example that might help #12712

Navigation Menu

Repository navigation

Support DictionaryString for Regex matching operators #12618

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	macro_rules! binary_string_array_flag_op {
	($LEFT:expr, $RIGHT:expr, $OP:ident, $NOT:expr, $FLAG:expr) => {{
	match $LEFT.data_type() {
	DataType::Utf8View \| DataType::Utf8 => {
	compute_utf8_flag_op!($LEFT, $RIGHT, $OP, StringArray, $NOT, $FLAG)
	},
	DataType::LargeUtf8 => {
	compute_utf8_flag_op!($LEFT, $RIGHT, $OP, LargeStringArray, $NOT, $FLAG)
	},
	other => internal_err!(
	"Data type {:?} not supported for binary_string_array_flag_op operation '{}' on string array",

Support DictionaryString for Regex matching operators #12618

Description

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Activity

alamb commented on Sep 25, 2024

goldmedal commented on Sep 28, 2024

blaginin commented on Oct 2, 2024

blaginin commented on Oct 2, 2024

alamb commented on Oct 3, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions