Skip to content

Add support for string view to a few functions #13330

Closed

Description

@timsaucer

Is your feature request related to a problem or challenge?

These functions are failing when passing in a string view instead of a string. Now that string view is the default type, it is causing some regressions on existing code that have to be modified.

Describe the solution you'd like

Add options of passing in string view to these:

  • ArrayToString
  • NamedStructFunc
  • DateBinFunc (I couldn't quite track down where the Utf->Interval or Timestamp conversion occurs
  • BitLengthFunc
  • ConcatFunc (gives error on optimizer not matching schema)
  • EncodeFunc

Describe alternatives you've considered

No response

Additional context

These were discovered during the datafusion-python migration to DF43.0.0. This commit is where I made adjustments in that repo to account for these: apache/datafusion-python@fcb5f96

Activity

  1. timsaucer commented on Nov 10, 2024

    @timsaucer
    MemberAuthor

    Related: #13347

  2. Omega359 commented on Nov 11, 2024

    @Omega359
    Contributor

    regexp_match looks to not support stringview either.

    // TODO: Native support Utf8View for regexp_match.
                        TypeSignature::Exact(vec![Utf8, Utf8]),
                        TypeSignature::Exact(vec![LargeUtf8, LargeUtf8]),
                        TypeSignature::Exact(vec![Utf8, Utf8, Utf8]),
                        TypeSignature::Exact(vec![LargeUtf8, LargeUtf8, LargeUtf8]),
    
  3. Omega359 commented on Nov 11, 2024

    @Omega359
    Contributor

    regexp_match filed as #13357 - was a duplicate of already filed #11911

  4. timsaucer commented on Nov 11, 2024

    @timsaucer
    MemberAuthor

    I'm going to recommend a 43.1.0 release after we get all these resolved

  5. Omega359 commented on Nov 11, 2024

    @Omega359
    Contributor

    array_to_string also does not appear to support Utf8View

  6. Omega359 commented on Nov 11, 2024

    @Omega359
    Contributor

    None of the crypto functions support UTF8View directly as a input type. For example digest:

    signature: Signature::one_of(
                    vec![
                        Exact(vec![Utf8, Utf8]),
                        Exact(vec![LargeUtf8, Utf8]),
                        Exact(vec![Binary, Utf8]),
                        Exact(vec![LargeBinary, Utf8]),
                    ],
                    Volatility::Immutable,
                ),
  7. Omega359 commented on Nov 11, 2024

    @Omega359
    Contributor

    NullIf:

    static SUPPORTED_NULLIF_TYPES: &[DataType] = &[
        DataType::Boolean,
        DataType::UInt8,
        DataType::UInt16,
        DataType::UInt32,
        DataType::UInt64,
        DataType::Int8,
        DataType::Int16,
        DataType::Int32,
        DataType::Int64,
        DataType::Float32,
        DataType::Float64,
        DataType::Utf8,
        DataType::LargeUtf8,
    ];
  8. Omega359 commented on Nov 11, 2024

    @Omega359
    Contributor

    nvl:

    static SUPPORTED_NVL_TYPES: &[DataType] = &[
        DataType::Boolean,
        DataType::UInt8,
        DataType::UInt16,
        DataType::UInt32,
        DataType::UInt64,
        DataType::Int8,
        DataType::Int16,
        DataType::Int32,
        DataType::Int64,
        DataType::Float32,
        DataType::Float64,
        DataType::Utf8,
        DataType::LargeUtf8,
    ];
  9. Omega359 commented on Nov 14, 2024

    @Omega359
    Contributor

    crypto PR - #13407
    array_to_string/string_to_array PR - #13403
    nvl PR - #13382
    nullif PR - #13380

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions