Skip to content

cold-store-reaper: 700 GB/day upload hard stop and 3 TiB budget are reported by status but never enforced in upload / put-backup #938

Closed

Description

@dezren39

Finding

cold-store-reaper's 700 GB/day upload hard stop and 3 TiB project budget are reported but never enforced. status computes hard_stop_ok / within_budget, but no upload path checks them before sending bytes, so the "hard stop" stops nothing.

Found by the pr-diff-review job reviewing PR #923 (cold-store-reaper v1) and PR #929 (v2), 2026-10-25.

Evidence

  • pkgs/cold-store-reaper/schema.sql:56-57 seeds the guardrails: project_budget_bytes = 3298534883328 (3 TiB) and daily_upload_hard_stop_bytes = 700000000000 (700 GB). PR cold-store-reaper v1: Google Drive cold-store CLI (pack / resumable upload+verify / range fetch) #923's body describes the manifest as holding the "700 GB/day upload hard stop".
  • On current main (92d0c6ee), pkgs/cold-store-reaper/cold_store.py:
    • upload() (line 558) goes straight from the bundle-state check to _resumable_upload(...) — no read of the transfer ledger, daily_upload_hard_stop_bytes, or project_budget_bytes.
    • put_backup() (line 672, added by PR cold-store-reaper v2: D1 manifest sync, DB-backup class, rehydrator Worker #929) likewise uploads via _resumable_upload(...) (line ~693) with no guardrail check, then writes its backup-upload ledger row afterwards.
    • The only consumers of the two meta values are in status_data() (lines 888-901), which derives hard_stop_ok and within_budget for display. grep -n hard_stop cold_store.py shows no other use.
  • So a loop or operator error that uploads past 700 GB in 24h (Google Drive's own daily upload ceiling — hitting it mid-run strands resumable sessions) or past the 3 TiB project budget is stopped by nothing in the tool that owns those numbers.

Why it matters

The design's cost/quota guardrail exists only as a status readout. The reaper is meant to run unattended over very large archives; an unenforced hard stop converts a designed safety limit into a false sense of security, and the first notice would be Drive quota failures or budget overrun discovered after the fact.

Acceptance criteria

  • upload and put-backup refuse to start (clear ColdStoreError naming the limit and the current 24h/project totals) when the transfer would cross daily_upload_hard_stop_bytes (last-24h uploads + this transfer) or project_budget_bytes (verified cold bytes + this transfer).
  • The check uses the same ledger/meta accounting as status_data so status and enforcement can never disagree.
  • Tests (fake-Drive, following the existing test style) cover: upload refused at/over the daily stop, upload allowed under it, put-backup refused likewise.
  • README's description of the hard stop / budget matches the enforced behavior.

— filed by the pr-diff-review job (PRs #923, #929); triage via the repo-health queue pilot.

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    task/triageQueue packet kind: verify, reproduce, and route this item
    priority/normalOrdinary priority (the default)
    queue/queuedHas a queued task in the Supabase agent queue
    app/cold-store-reaperPackage/app: cold-store-reaper (pkgs/ or apps/ in this repo)
    on Oct 26, 2026
  2. dezren39 commented on Oct 26, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · filed

  3. dezren39 commented on Oct 26, 2026

    @dezren39
    MemberAuthor

    Fixed in PR #988 (merged, merge commit e9d2a6847bf674116ed59aa7bf0fe3b547aaef5c).

    • The ledger/meta accounting status used for display was extracted into ColdStore._upload_accounting() — rolling-24h transfer_ledger upload bytes, verified cold bytes (sum of verified bundles), and the daily_upload_hard_stop_bytes / project_budget_bytes meta values. status_data() consumes it too, so readout and enforcement share one implementation and cannot disagree (acceptance criterion 2).
    • New _check_upload_limits(conn, nbytes, what) refuses with a ColdStoreError naming the limit, the current usage, and the refused size when uploads_24h + transfer > hard stop, or cold bytes + transfer > budget.
    • Enforcement points: upload() (before the bundle is marked uploading) and put_backup() (before the folder lookup, so a refusal causes zero Drive writes). Note: no write path enforced the limits before — the bundle path included — so the helper is new, shared code rather than a reuse of an existing check.
    • Tests (TestUploadLimits): upload refused at the daily stop with zero Drive writes observed (no session/file/folder; bundle stays sealed), upload refused when the rolling ledger alone crosses the stop, put-backup refused at the project budget, and just-under-limit upload + put-backup succeed. README gained an "Upload guardrails (enforced, not just reported)" section.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/cold-store-reaperPackage/app: cold-store-reaper (pkgs/ or apps/ in this repo)kind/bugSomething isn't working (prefixed form of the classic bug label)priority/normalOrdinary priority (the default)queue/queuedHas a queued task in the Supabase agent queuesource/pr-diff-reviewFiled by the pr-diff-review jobstatus/needs-triageNot yet triaged; needs a first looktask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions