[Bug] [Python] fast_forward keeps main's own newer snapshots #10887

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

pypaimon 2.1.0. fast_forward is unchanged on master.

Compute Engine

Python API (pypaimon 2.1.0, Python 3.12.16, pyarrow 19.0.1) on a local filesystem catalog. Spark 3.5.6 with paimon-spark-3.5_2.12 2.1.0 for the comparison.

Minimal reproduce step

Currently, pypaimon's fast_forward copies the branch's snapshot files into main without first deleting main's snapshots from the branch point on, so once main has more snapshots than the branch, main keeps reading its own. Spark's CALL sys.fast_forward on the same table gives [1, 2].

import pyarrow as pa
from pypaimon import CatalogFactory, Schema

cat = CatalogFactory.create({"warehouse": "wh"})
cat.create_database("db", True)
sch = pa.schema([("k", pa.int64())])
cat.create_table("db.t", Schema.from_pyarrow_schema(sch), False)


def write(name, k):
    wb = cat.get_table(name).new_batch_write_builder()
    w, c = wb.new_write(), wb.new_commit()
    w.write_arrow(pa.table({"k": [k]}, schema=sch))
    c.commit(w.prepare_commit())


def read(name):
    rb = cat.get_table(name).new_read_builder()
    return sorted(rb.new_read().to_arrow(rb.new_scan().plan().splits())["k"].to_pylist())


write("db.t", 1)
cat.get_table("db.t").create_tag("v1", 1)
cat.create_branch("db.t", "b", "v1")
write("db.t$branch_b", 2)
write("db.t", 3)
write("db.t", 4)
cat.fast_forward("db.t", "b")
print(read("db.t"))
# [1, 3, 4], should be [1, 2]
print(read("db.t$branch_b"))
# [1, 2]

What doesn't meet your expectations?

After fast_forward("db.t", "b") main holds the branch's state and reads [1, 2].

Anything else?

If the branch has added a column, reading main after fast_forward raises AttributeError: 'NoneType' object has no attribute 'fields', since the branch's schema files are not copied either.

Are you willing to submit a PR?

• I'm willing to submit a PR!