[Feature] Support Spark expression: json_tuple #3160

What is the problem the feature request solves?

Note: This issue was generated with AI assistance. The specification details have been extracted from Spark documentation and may need verification.

Comet does not currently support the Spark json_tuple function, causing queries using this function to fall back to Spark's JVM execution instead of running natively on DataFusion.

JsonTuple is a generator expression that extracts multiple fields from a JSON string and returns them as separate columns in a tuple format. It takes a JSON string as the first argument followed by one or more field path expressions, returning a row with the extracted values in the order specified.

Supporting this expression would allow more Spark workloads to benefit from Comet's native acceleration.

Describe the potential solution

Spark Specification

Syntax:

json_tuple(json_string, field1, field2, ...)

Arguments:

Return Type: Returns a StructType with columns named c0, c1, c2, etc., where each column corresponds to the extracted field values. All output columns have nullable string type matching the input JSON expression's data type.

Supported Data Types:
Input arguments must be string types with collation support (specifically StringTypeWithCollation with trimming support). All children expressions must accept string data types.

Edge Cases:

• Always returns at least one row (nullable = false), even for invalid JSON
• Null JSON input results in null values for all extracted fields
• Invalid or missing field paths return null for those specific columns
• Foldable (constant) field expressions are pre-evaluated for performance optimization
• Requires minimum 2 arguments (JSON string + at least one field name), throws QueryCompilationErrors.wrongNumArgsError otherwise

Examples:

-- Extract name and age from JSON
SELECT json_tuple('{"name":"John", "age":30, "city":"NYC"}', 'name', 'age') AS (name, age);
-- Returns: John, 30

-- Extract nested fields
SELECT json_tuple('{"user":{"id":123,"email":"test@example.com"}}', 'user.id', 'user.email') AS (id, email);

// DataFrame API usage
import org.apache.spark.sql.functions._
df.select(json_tuple(col("json_col"), lit("field1"), lit("field2")).as(Seq("c0", "c1")))

Implementation Approach

See the Comet guide on adding new expressions for detailed instructions.

1. Scala Serde: Add expression handler in spark/src/main/scala/org/apache/comet/serde/
2. Register: Add to appropriate map in QueryPlanSerde.scala
3. Protobuf: Add message type in native/proto/src/proto/expr.proto if needed
4. Rust: Implement in native/spark-expr/src/ (check if DataFusion has built-in support first)

Additional context

Difficulty: Large
Spark Expression Class: org.apache.spark.sql.catalyst.expressions.JsonTuple

Related:

• get_json_object - Extract single JSON field
• from_json - Parse JSON with explicit schema
• json_extract - Alternative JSON field extraction

This issue was auto-generated from Spark reference documentation.