Skip to content

older_than reads a naive datetime as UTC and expires snapshots it should keep #4182

Open

Description

@RyanLin5967

Apache Iceberg version

0.12.0 (latest release)

Please describe the bug 🐞

Currently, older_than converts its cutoff with datetime_to_millis, which takes a naive datetime as UTC, so in Tokyo the docs' older_than(datetime.now() - timedelta(days=3)) keeps only 63 hours of snapshots. An aware datetime is read correctly.

import os, time, tempfile
os.environ["TZ"] = "Asia/Tokyo"  # UTC+9, any zone east of UTC does it
time.tzset()

from datetime import datetime, timedelta
import pyarrow as pa
from pyiceberg.catalog.sql import SqlCatalog

cat = SqlCatalog("c", uri="sqlite:///:memory:", warehouse=f"file://{tempfile.mkdtemp()}")
cat.create_namespace("ns")
t = cat.create_table("ns.t", schema=pa.schema([("x", pa.int64())]))
for x in (1, 2, 3):
    t.append(pa.table({"x": [x]}))

t.maintenance.expire_snapshots().older_than(datetime.now() - timedelta(hours=1)).commit()
print(len(cat.load_table("ns.t").snapshots()))
# 1, should be 3
# all three snapshots are seconds old, so none is older than an hour

West of UTC the cutoff lands early instead, so it keeps snapshots it should expire.

tested on pyiceberg 0.12.0 and main

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time

Activity

  1. Rahul-cs50 commented on Oct 31, 2026

    @Rahul-cs50

    I'd like to work on this one; I'll open a PR shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions