You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
inspect.files() and the other file metadata tables raise ArrowInvalid when a snapshot has no manifests #4269
Appending an empty Arrow table creates a snapshot whose manifest list has no manifests. After that, inspect.files(), data_files(), delete_files(), all_files(), all_data_files(), and all_delete_files() all raise ArrowInvalid: Must pass at least one table. I expected each of them to return an empty table with the files schema.
importtempfileimportpyarrowaspafrompyiceberg.catalog.sqlimportSqlCatalogwarehouse=tempfile.mkdtemp()
catalog=SqlCatalog("t", uri=f"sqlite:///{warehouse}/catalog.db", warehouse=f"file://{warehouse}")
catalog.create_namespace("n")
df=pa.table({"x": pa.array([], pa.int64())})
table=catalog.create_table("n.t", schema=df.schema)
table.append(df)
table.inspect.files() # ArrowInvalid: Must pass at least one table
Traceback (most recent call last):
File "repro.py", line 12, in <module>
table.inspect.files() # ArrowInvalid: Must pass at least one table
^^^^^^^^^^^^^^^^^^^^^
File "pyiceberg/table/inspect.py", line 900, in files
return self._files(snapshot_id)
^^^^^^^^^^^^^^^^^^^^^^^^
File "pyiceberg/table/inspect.py", line 888, in _files
return pa.concat_tables(results)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/table.pxi", line 6344, in pyarrow.lib.concat_tables
File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Must pass at least one table
overwrite() with an empty table after a non-empty append also leaves a snapshot with no manifests, and files() fails the same way.
_files reads one table per manifest and ends with pa.concat_tables(results), so an empty manifest list calls pa.concat_tables([]). _all_files does the same at L981. Both already return an empty table one level up: _files when the table has no current snapshot (L876-L877), and _all_files when the table has no snapshots (L968-L970). Could they return the same empty table when there are no manifests?
Activity