Skip to content

cold-store-reaper: S3 list-page parser trusts the body — non-XML/wrong-root pages and bad Size values crash raw or read as an empty bucket #2733

Closed

Description

@dezren39

Summary

cold-store-reaper's S3 ListObjectsV2 parser (_parse_s3_list_page) trusts the response body completely: it never validates that the XML root is a list result, never converts XML parse failures to the module's ColdStoreError, parses Size with a bare int(...), and accepts negative sizes. A malformed or shape-drifted R2/proxy response therefore either crashes the CLI with a raw traceback, or — worse — is silently read as an empty/negative-sized listing that downstream backup and eviction code trusts.

Evidence (main ba5229e9, pkgs/cold-store-reaper/cold_store.py)

  • _parse_s3_list_page (:138) runs ET.fromstring(body) at :146 with no guard. main() catches only ColdStoreError (end of main, ~:2240), so a non-XML body escapes as a raw xml.etree.ElementTree.ParseError traceback.
  • Size is parsed at :176 as int(fields.get("Size") or 0) with no guard: "abc" / "1.5" raise a raw ValueError; "-5" is accepted and stored as a negative object size.
  • The root element is never checked: any well-formed XML that is not a ListBucketResult (e.g. an HTML/XML error or maintenance page returned with HTTP 200 — _s3_check at :1557 only rejects status >= 300) yields ([], False, None), i.e. "the bucket is empty", from s3_list_objects (:1563, parse call :1574).
  • The parsed sizes are load-bearing downstream: backup_supabase passes newest["size"] as the byte-count verification for a downloaded backup (:1789), and evict_apply writes entry["size"] into the transfer ledger (:1915); evict_scan (:1817) plans deletions off this same listing.

Verified this run with the real module (md5-matched copy of main's cold_store.py; probe on the laptop at ~/git/fleet/probe-2026-10-30-laneB-s3parse/probe.py, stdlib-only):

  • control: valid ListBucketResult page → [{'key': 'a.bin', 'size': 3, 'last_modified': 2026-10-01T00:00Z}], truncated False (OK)
  • body {"error": "nope"} (non-XML) → RAW ParseError: not well-formed (invalid token)
  • <Size>abc</Size> / <Size>1.5</Size> → RAW ValueError: invalid literal for int()
  • <Size>-5</Size> → accepted, object size -5
  • well-formed <html><body>proxy error</body></html> → silently [], truncated False (read as an empty bucket)

Why it matters

This listing is the input to backup-supabase and to eviction, which deletes objects. The module's own contract elsewhere is fail-closed with named ColdStoreErrors (transport retries, the #2110 aware-or-None datetime rule, the curl size verification). The list parser is the one place a drifted body can either mask the real failure behind an unrelated traceback or, in the wrong-root case, present a non-empty bucket as empty to code that plans from it.

Acceptance criteria

  • _parse_s3_list_page raises ColdStoreError (naming the problem) for a body that is not well-formed XML and for a well-formed document whose root is not a list result.
  • A Contents entry whose Size is missing keeps the current default; a Size that is non-numeric or negative raises ColdStoreError instead of a raw ValueError or a negative object size.
  • Regression tests cover: non-XML body, wrong-root XML body, Size = abc, 1.5, -5, plus the valid-page control (including the existing pagination and LastModified behaviour).
  • Existing cold-store-reaper tests pass unchanged.

Dedup

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    priority/normalOrdinary priority (the default)
    queue/queuedHas a queued task in the Supabase agent queue
    source/creatorFiled by the new-issue-creator job
    task/triageQueue packet kind: verify, reproduce, and route this item
    app/cold-store-reaperPackage/app: cold-store-reaper (pkgs/ or apps/ in this repo)
    effort/xsTrivial: expected under 30 minutes of work
    on Oct 30, 2026
  2. dezren39 commented on Oct 30, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · filed

    • Agent: new-issue-creator (new-issue-creator-code-b)
    • Session / ID: cron job new-issue-creator-code-b, run 2026-10-30T12:20Z
    • Found via: filed this run
    • Task: Triage the S3 list-page parser hardening: root/parse validation to ColdStoreError, Size shape gate, regression tests.
    • Expectation: effort/xs — triage pins the exact error shape, then routes a small_fix packet; done when a fix with the listed regression cases merges.
    • Links: probe (laptop) ~/git/fleet/probe-2026-10-30-laneB-s3parse/; body file hidden_files/issue-code-s3parse-body.md in the goal workspace
  3. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Oct 31, 2026
  4. 23 remaining items

  5. added
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/claimedQueue worker lifecycle state
    on Oct 31, 2026
  6. dezren39 commented on Oct 31, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    event: parked
    task: 4414 (lane F)
    artifact: PR #3133 @ 95bf268
    state: Codex clean on head (re-verified). Dispatched nix-ci run 41224029839 on the branch — still QUEUED after 5-min poll (fleet backlog). Laptop suite 101 tests OK per opening lane.
    resume: run green -> merge -> #2733 auto-closes -> complete fixed_merged.

  7. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Oct 31, 2026
  8. dezren39 commented on Oct 31, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

    • Agent: queue worker (repo-worker-code), lane C
    • Session / ID: queue task 4414 (resume claim, lane C shift 2026-10-31 13:14 CDT)
    • Found via: queue claim (resume at awaiting_ci)
    • Task: PR 3133 re-verified: nix-ci run 41224029839 now completed SUCCESS at head 95bf268 (Nix flake evaluation + proxy port fallbacks green), Codex review state clean (no findings, none unresolved), no unresolved review threads, PR MERGEABLE/CLEAN. Merging per standing criteria.
    • Links: fix(cold-store-reaper): fail closed on malformed S3 list pages (#2733) #3133
  9. dezren39 commented on Oct 31, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/cold-store-reaperPackage/app: cold-store-reaper (pkgs/ or apps/ in this repo)effort/xsTrivial: expected under 30 minutes of workkind/bugSomething isn't working (prefixed form of the classic bug label)priority/normalOrdinary priority (the default)source/creatorFiled by the new-issue-creator jobtask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions