Skip to content

[rust] Add reusable UnionRead source boundaries #4886

Open

Description

@naivedogger

Parent: #4713

Bounded UnionRead needs a readable lake snapshot and the complementary Fluss log ranges. The default Rust reader and engine-native integrations should be able to share these boundaries without opening a lake catalog or planning lake files.

Introduce fluss-lake with FlussLakeTable::prepare() and a serializable FlussLakeReadContext.

Scope:

  • Capture table/schema identity, physical lake table mapping, merge engine configuration, and an optional readable lake snapshot.
  • Capture half-open log ranges [start, stop) for every live bucket, using each partition’s actual bucket count.
  • Support lake-enabled tables without a lake snapshot by requiring log history from offset zero.
  • Preserve the required start offset separately from the observed earliest retained offset, so callers can detect missing history after pruning.
  • Validate context versions, identities, bucket coverage, and compatibility with live table metadata. Keep credentials and query configuration outside the context.

The context is independent of projection and filters. It is not a global transactional snapshot, a retention lease, or a physical execution plan. Its live log ranges are not a complete inventory of lake partitions.

Lake planning, Paimon reading, and Append/PK execution are covered by subsequent tasks. Integration-test CI wiring will follow separately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions