You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
A retried append records its file under a concurrently added partition spec #4188
Currently, a commit retry writes the added manifest under the refreshed default spec, so a file partitioned by p gets its p value recorded as q and filters on q prune it. A handle loaded after the spec change is correct.
importtempfileimportpyarrowaspafrompyiceberg.catalog.sqlimportSqlCatalogcat=SqlCatalog("c", uri="sqlite:///:memory:", warehouse=f"file://{tempfile.mkdtemp()}")
cat.create_namespace("ns")
row=lambdax, p, q: pa.table({"x": [x], "p": [p], "q": [q]})
t=cat.create_table("ns.t", schema=row(1, 5, 50).schema)
witht.update_spec() asu:
u.add_identity("p")
b=cat.load_table("ns.t")
withb.update_spec() asu:
u.remove_field("p")
u.add_identity("q")
b.append(row(2, 6, 60))
t.append(row(3, 7, 70))
print(cat.load_table("ns.t").scan(row_filter="q == 70").to_arrow()["x"].to_pylist())
# [], should be [3]# t's commit retried after b's append, and its file was recorded under b's spec as q=7
If the new spec has a different number of fields, the append raises IndexError, and dynamic_partition_overwrite mislabels its file the same way.
tested on pyiceberg 0.12.0 and main
Willingness to contribute
I can contribute a fix for this bug independently
I would be willing to contribute a fix for this bug with guidance from the Iceberg community
I cannot contribute a fix for this bug at this time
Activity
tanmayrauth commentedon Oct 30, 2026
Can I work on this?