cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies

Problem

pkgs/cold-store-reaper/cold_store.py — put_backup() (lines 993–1027) files a DB dump into Drive and records it as stored, but never verifies what Drive actually received. The bundle path in the same file does verify: upload() (lines 908–930) reads back files.get md5Checksum + size and refuses to mark the bundle verified on a mismatch. put_backup() has no equivalent step:

file_id = self._resumable_upload(src, name, folder_id) (line 1019)
a transfer_ledger row upload / backup-upload is written immediately (lines 1020–1024)
the result dict is returned as success

That ledger row is load-bearing: _upload_accounting() (lines 497–541) computes the project-budget backup bytes from backup-upload rows, and backup_supabase() / backup_d1() / scripts/export-turso.sh all file through put_backup(). A backup whose stored bytes differ from the local dump (truncated/corrupted server-side assembly, wrong content) is counted as safely filed, and the corruption is only discovered at restore time — for a system whose entire purpose is disaster recovery of the estate's databases (d1, supabase, turso dumps, nightly via scripts/nightly-backup.sh).

Evidence (verified this run)

Laptop probe ~/git/fleet/probe-2026-10-31-laneB-backupverify/probe.py against the real cold_store.py (sha256 fba42e08027514abae2de1602e755854ebfd6a3b578a5eece34b6c0becd39b95, identical on the sandbox and laptop clones, main 357ec7fb):

_resumable_upload stubbed to return a file id while Drive holds different bytes, and cold_store.http_request rigged to raise if any verification request is attempted.
Result: put_backup("d1", dump.sql) returned success ({'db': 'd1', 'name': 'd1-20261031T212413Z-dump.sql', 'drive_file_id': 'file-corrupt-1', 'bytes': 2000}), zero http_request calls were made during the call, and a transfer_ledger row upload / 2000 / backup-upload was written for the full source size.

So there is provably no read-back of any kind on the backup path — not md5, not size, not even a files.get.

Why it matters

The bundle path's verification exists precisely because an unverified upload cannot be trusted; DB backups are at least as critical as bundle objects (they are the recovery source for the estate's databases, and the restore drill in scripts/restore-drill.sh exercises d1 backups).
Budget accounting treats the unverified row as stored bytes, so the failure is invisible in status as well.
The fix pattern already exists in the same file (upload() lines 908–930) and the test fake Drive already serves files.get with md5Checksum/size for the bundle tests.

Acceptance criteria

After _resumable_upload, put_backup() reads back the uploaded file's md5Checksum and size (Drive files.get, fields id,md5Checksum,size) and compares them to the local file's md5 (md5_file) and size.
On any mismatch, put_backup() raises ColdStoreError naming the mismatch and writes no backup-upload ledger row (mirroring upload()'s "bundle NOT marked verified" behaviour).
On match, behaviour is unchanged: same dated name, same ledger row, same result fields.
A regression test in pkgs/cold-store-reaper/tests/test_cold_store.py covers the mismatch case (fake Drive reporting a different md5/size → error, no ledger row) and the existing backup tests still pass (python3 -m unittest discover -s pkgs/cold-store-reaper/tests).

Filed by new-issue-creator-code-b, 2026-10-31. Distinct from #1427 (CI coverage for this package, closed today) — this is a runtime verification gap in put_backup itself.

🤖 Agent note · filed

Agent: new-issue-creator-code-b
Session / ID: new-issue-creator-code-b cron run 2026-10-31 16:20 CDT
Found via: filed this run — read put_backup() (cold_store.py:993-1027) against the bundle upload() verification path (cold_store.py:908-930) and verified the gap with a laptop probe driving the real module (sha256-matched on both clones): _resumable_upload stubbed to a corrupt Drive copy, http_request rigged to raise on any verification attempt — put_backup still returned success, made zero verification calls, and wrote the full-size backup-upload ledger row.
Task: Filed this issue: put_backup files DB dumps (d1/supabase/turso, nightly) with no md5/size read-back, while bundle upload() in the same file verifies both and refuses to mark verified on mismatch.
Expectation: effort s; done when put_backup verifies md5+size after upload and writes no ledger row on mismatch, pinned by a regression test in pkgs/cold-store-reaper/tests/test_cold_store.py.
Links: Evidence body: hidden_files/issue-code-coldstore-putbackup-verify-body.md · Probe: laptop ~/git/fleet/probe-2026-10-31-laneB-backupverify/ · Distinct: ci(cold-store-reaper): no workflow runs the package's test suites — 2,700+ lines of tests execute only when a human remembers #1427 (CI coverage, closed)

🤖 Agent note · parked

Agent: queue worker (repo-worker-code), lane C shift
Session / ID: task 5674
Found via: queue claim (resumed at stage awaiting_ci)
Task: Parked 20m on CI pending. This shift dispositioned the Codex P2 finding on PR fix(cold-store): verify backup uploads (#3230) #4330 (unledgered uploads left in Drive after verification failure): fixed in commit f4351b8 — verification failures now trash the uploaded file before raising, with 3 new regression tests; full cold-store suite passes on the laptop (121 tests, 1 skipped); the Codex thread has the disposition reply. PR fix(cold-store): verify backup uploads (#3230) #4330's two required checks remain QUEUED after a 5-minute poll (runner backlog).
Expectation: Resume when PR fix(cold-store): verify backup uploads (#3230) #4330's checks complete: check for a fresh Codex round on head f4351b8, merge when green, close issue cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230, complete task 5674.
Links: PR fix(cold-store): verify backup uploads (#3230) #4330

agent-note:v1 — picked up
Agent: repo-worker-code (OpenChamber scheduled lane)
Session: ses_e5e63b9e2c2ffeNWcJTcGcVvwB · queue task 5674
Task: Resume verification gate for the existing cold-store backup PR.
Fresh state: #3230 OPEN; PR #4330 OPEN at f4351b8 and authored by dezren39. Required checks remain QUEUED; CodeRabbit skipped. No files or PR content changed.
Next: park pending checks; revalidate PR/checks/review and ownership before any merge action.

agent-note:v1 — parked
Agent: repo-worker-code (OpenChamber scheduled lane)
Session: ses_e5e63b9e2c2ffeNWcJTcGcVvwB · queue task 5674
Verified: #3230 OPEN; PR #4330 OPEN at f4351b8, authored by dezren39. Checks python: unittest suites and worker: local proof are QUEUED in run 41440146673; CodeRabbit skipped.
Reason: CI pending; PR is Drew-owned and must not be modified or merged by this lane. No files or PR content changed.
Resume: after checks complete, freshly revalidate PR/checks/review and ownership. Queue parked 20 minutes; work/assigned stays set.

🤖 Agent note · parked

Agent: queue worker (repo-worker-code), lane A
Session / ID: task 5674
Found via: queue claim
Task: Cold-store put_backup read-back verification for issue cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230 — implementation already on PR fix(cold-store): verify backup uploads (#3230) #4330; revalidated fresh.
Expectation: Parked 20m on CI. PR fix(cold-store): verify backup uploads (#3230) #4330 head f4351b8 is unchanged and mergeable; required checks python: unittest suites and worker: local proof are still QUEUED (runner backlog), CodeRabbit neutral. Resume: when both checks complete green, revalidate ownership/review and merge PR fix(cold-store): verify backup uploads (#3230) #4330, then complete task 5674; if red, fix on branch queue/5674-3230-drive-backup-verification.
Links: PR fix(cold-store): verify backup uploads (#3230) #4330

🤖 Agent note · state: parked

Agent: queue worker (repo-worker-code), Lane C shift
Session / ID: task 5674
Found via: queue claim (resume from awaiting_ci)
Task: Re-validated fresh at 14:17Z: issue cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230 OPEN, no hard-stop labels; PR fix(cold-store): verify backup uploads (#3230) #4330 OPEN, head f4351b8af unchanged. The Codex summary still flags an unresolved finding on this head, so I read the thread and the diff directly: the single Codex P2 (backups left in Drive after verification fails) was fixed in f4351b8af itself — all three verification-failure raises in put_backup route through a verification_failed helper that trashes the uploaded file before raising, with tests asserting the trash — and the fix is replied to on the finding thread. The finding is dispositioned; the unresolved flag is stale (no Codex re-review has marked the thread resolved). Checks (run 41440146673: python unittest suites, worker local proof) are still pending in the runner backlog — recorded suite evidence: 118 tests passed, 1 skipped. Parked 20 min per the CI rule.
Expectation: resume = checks green on PR fix(cold-store): verify backup uploads (#3230) #4330 → merge, close issue cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230, complete task 5674.
Links: PR fix(cold-store): verify backup uploads (#3230) #4330 · branch queue/5674-3230-drive-backup-verification

🤖 Agent note · picked up for ownership revalidation

Agent: queue worker (repo-worker-code), OpenChamber scheduled lane
Session / ID: OpenCode session ses_e5e6382f632ffeoXT3muAxJr0w · queue task 5674
Found via: atomic queue claim; resumed from awaiting_ci checkpoint
Fresh metadata: cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230 OPEN and assigned to dezren39; sole matching PR fix(cold-store): verify backup uploads (#3230) #4330 OPEN, non-draft, authored by dezren39 on the task branch.
Action: no PR checks/comments/content/diff/branch/worktree/source accessed or changed.
Next: park pending explicit ownership release or PR disposition.

🤖 Agent note · parked — Drew-owned PR remains open

Agent: queue worker (repo-worker-code), OpenChamber scheduled lane
Session / ID: OpenCode session ses_e5e6382f632ffeoXT3muAxJr0w · queue task 5674
Evidence: cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230 is OPEN and assigned to dezren39; sole matching PR fix(cold-store): verify backup uploads (#3230) #4330 is OPEN, non-draft, authored by dezren39.
Disposition: parked until 2026-11-02T14:25:00Z. No PR checks/comments/content/diff/branch/worktree/source accessed or changed. Existing issue labels reflect parked state: work/assigned, queue/parked, status/parked.
Resume condition: explicit Drew release/handoff or PR disposition; revalidate metadata before any further access.

🤖 Agent note · picked up for ownership revalidation

Agent: queue worker (repo-worker-code), OpenChamber scheduled lane
Session / ID: queue task 5674
Found via: atomic queue claim; resumed from prior parked checkpoint
Fresh metadata: cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230 OPEN and assigned to dezren39; sole matching PR fix(cold-store): verify backup uploads (#3230) #4330 OPEN, non-draft, authored by dezren39 on queue/5674-3230-drive-backup-verification, head f4351b8 unchanged.
Action: no PR checks/comments/content/diff/branch/worktree/source accessed or changed.
Next: park pending explicit ownership release or PR disposition.

🤖 Agent note · parked — Drew-owned PR remains open

Agent: queue worker (repo-worker-code), OpenChamber scheduled lane
Session / ID: queue task 5674
Evidence: cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230 is OPEN and assigned to dezren39; sole matching PR fix(cold-store): verify backup uploads (#3230) #4330 is OPEN, non-draft, authored by dezren39, head f4351b8 unchanged since prior check.
Disposition: parked 30 min. No PR checks/comments/content/diff/branch/worktree/source accessed or changed. Existing issue labels reflect parked state: work/assigned, queue/parked, status/parked.
Resume condition: explicit Drew release/handoff or PR disposition; revalidate metadata before any further access.

Merge pull request #4330 from developing-today/queue/5674-3230-drive-…

🤖 Agent note · done

Agent: queue worker (repo-worker-code), lane C
Session / ID: task 5674
Found via: queue claim
Task: Revalidated task 5674 for issue cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230 after PR fix(cold-store): verify backup uploads (#3230) #4330's disposition — the prior Drew-owned hard stop's resume condition.
Expectation: done = fix verified on main and issue closed
Links: PR fix(cold-store): verify backup uploads (#3230) #4330 MERGED 2026-11-02T15:29:14Z (merge commit 2474e1f). Verified fresh on main: pkgs/cold-store-reaper/cold_store.py put_backup now read-back verifies the Drive upload (files.get md5/size) and refuses + trashes the artifact before any ledger write on mismatch/failure — exactly the acceptance in issue cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230. PR fix(cold-store): verify backup uploads (#3230) #4330's body records the regression tests passing after the fix. Closing issue cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230 as completed; stale in-flight labels removed in this same action.

Navigation Menu

Repository navigation

cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230

Problem

Evidence (verified this run)

Why it matters

Acceptance criteria

48 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

cold-store-reaper: put_backup files DB dumps with no read-back verification — a corrupted Drive copy is ledgered as a good backup, unlike bundle upload which md5-verifies #3230

Description

Problem

Evidence (verified this run)

Why it matters

Acceptance criteria

Activity

dezren39 commented on Oct 31, 2026

48 remaining items

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

dezren39 commented on Nov 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions