Skip to content

[Bug] [Spark] Scan output ordering fails for special primary-key names #10657

Closed

Description

@liujiwen-up

Problem and environment

PaimonScan.outputOrdering passes raw primary-key names to Expressions.identity. Spark parses this argument as a multipart SQL identifier, so valid top-level names containing # or a backtick cause a ParseException, and a name such as a.b is interpreted as a nested field and causes an AnalysisException.

This differs from #5439: that fix quotes the bucket expression, but not the primary-key identity expressions used for output ordering.

Minimal reproduction

Configure a Paimon catalog named paimon, then run:

SET spark.sql.sources.v2.bucketing.enabled=true;
SET spark.paimon.scan.preserve-data-grouping=true;
CREATE TABLE paimon.default.ordering_repro (
  seq INT,
  `#event_id` STRING
) USING paimon
TBLPROPERTIES (
  'primary-key'='seq,#event_id',
  'bucket-key'='seq',
  'bucket'='1'
);
INSERT INTO paimon.default.ordering_repro VALUES (1, 'b'), (1, 'a'), (2, 'a');
SELECT seq, `#event_id` FROM paimon.default.ordering_repro;

Expected: the three inserted rows. Actual: query planning fails in Expressions.identity -> PaimonScan.outputOrdering with a syntax error at #.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions