Skip to content

aggregator(snapshots): isSnapshotRollup re-admits the shapes isSnapshotRecord was hardened to refuse — impossible dates, negative/string limit, unparseable resetsAt, inverted pairs expand into the reading path unchecked #3365

Closed

Description

@dezren39

Bug

SnapshotStore.readAll has two doors into the same reading path, and only one of them is hardened. Raw lines must pass isSnapshotRecord (pm-stack/aggregator/src/snapshots.ts:296), which was deliberately tightened (#2264/#2483): timestamps must be real ISO calendar dates via the strict isIsoDate primitive (because Date.parse rolls 2026-02-30 into March), limit must be a non-negative finite number when present, resetsAt must be a real ISO date when present, and pair values must have finite endpoints with low <= high.

Rollup lines take the other door: readAll gates them only on isSnapshotRollup (snapshots.ts:82, used at snapshots.ts:552) and then expandRollup (snapshots.ts:226) emits records straight into the reading path — the expanded records are never passed through isSnapshotRecord. But isSnapshotRollup does not apply the record-side rules:

  • Timestamps (firstTs/lastTs/minTs/maxTs) are checked with Number.isNaN(Date.parse(f)) (snapshots.ts:94-95) — exactly the loose check the record validator was fixed away from. 2026-02-30T00:00:00.000Z passes.
  • limit and resetsAt are not checked at all, even though expandRollup copies them onto the expanded last record and the record-side comment says these fields "are dereferenced downstream (pct-of-limit, reset comparisons), so a non-finite/negative limit or an impossible resetsAt date must refuse the record here, at the boundary."
  • Pair values (valueMin/valueMax/valueLast) accept any {low, high} with typeof === "number" endpoints (snapshots.ts:97-103): an inverted pair {low: 10, high: 5} passes, while the record side explicitly refuses inverted pairs.

The same loose gate also seeds compaction: existing rollups are re-aggregated through isSnapshotRollup at snapshots.ts:707, so a malformed rollup line is not just read, it is carried forward into the next compaction.

Evidence (verified this run, main 1a68d6a)

Laptop probe ~/git/fleet/probe-2026-11-01-laneA-rollupgate/probe.ts driving the real snapshots.ts (sha256 c6685035… matched on both clones; output in probe.txt in the same directory):

  • Control: a valid rollup passes isSnapshotRollup, expands to 3 records, and all 3 pass isSnapshotRecord.
  • lastTs = "2026-02-30T00:00:00.000Z" → isSnapshotRollup=true; one expanded record fails isSnapshotRecord.
  • limit = -5 → passes; the expanded last record (the one carrying limit) fails isSnapshotRecord.
  • limit = "100" (string) → passes; expanded last record fails isSnapshotRecord.
  • resetsAt = "not-a-date" → passes; expanded last record fails isSnapshotRecord.
  • valueMin = {low: 10, high: 5} (unit usd-est) → passes; expanded min record fails isSnapshotRecord.

Rollup files are kept indefinitely (13-month raw retention, rollups forever), so a single malformed or hand-edited rollup line poisons the reading path permanently, where a malformed raw line would only be dropped.

Minor, same function: the month check (/^\d{4}-\d{2}$/, snapshots.ts:86) accepts 2026-13 / 2026-00, months no writer produces.

Why it matters

The snapshot store is the usage hub's history: latest-value readers, pct-of-limit computations, and reset markers all consume readAll output trusting that the boundary validators refused impossible data. The record side of that boundary was hardened after real incidents (#2264/#2483); the rollup side silently re-admits the same shapes, and compaction re-seeds them forward, so the corruption is self-perpetuating rather than one-off.

Acceptance criteria

  • isSnapshotRollup refuses a rollup whose firstTs/lastTs/minTs/maxTs (and resetsAtTs/limitTs when present) are not strict ISO calendar dates (same isIsoDate primitive as isSnapshotRecord).
  • isSnapshotRollup refuses a present non-null limit that is not a non-negative finite number, and a present non-null resetsAt that is not a strict ISO date — mirroring the record-side rules.
  • isSnapshotRollup refuses pair values with non-finite endpoints or low > high, mirroring the record side.
  • month refuses impossible months (00, 13+).
  • Tests pin each refused shape above plus a valid rollup (including a valid pair-valued rollup and one carrying limit/resetsAt) still passing and expanding to records that all pass isSnapshotRecord.
  • No change to the compaction writer's output format or to expandRollup's expansion semantics for valid rollups.

Distinct from #2568 (recordsFromProviderUsage used_percent range, writer side) and #3295/#3285 (snapshot script argument handling). Found by comparing the two store-read validators in snapshots.ts.

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    priority/lowNice to have; do when convenient
    queue/queuedHas a queued task in the Supabase agent queue
    source/creatorFiled by the new-issue-creator job
    task/triageQueue packet kind: verify, reproduce, and route this item
    app/aggregatorPackage/app: aggregator (pm-stack in this repo)
    effort/sSmall: expected under 2 hours of work
    on Nov 1, 2026
  2. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · filed

    • Agent: new-issue-creator job (new-issue-creator-code-a)
    • Session / ID: cron job new-issue-creator-code-a, run 2026-11-01 00:35 CDT
    • Found via: filed this run
    • Task: Tighten isSnapshotRollup (pm-stack/aggregator/src/snapshots.ts:82) to the record-side boundary rules so malformed rollup lines cannot expand into the reading path unchecked.
    • Expectation: effort/s — done when the acceptance criteria on this issue pass, including tests pinning each refused shape and a valid-rollup control whose expansion passes isSnapshotRecord.
    • Links: evidence in the issue body; laptop probe at ~/git/fleet/probe-2026-11-01-laneA-rollupgate/ (probe.ts + probe.txt, real snapshots.ts, sha256 c6685035 matched on both clones, main 1a68d6a).
  3. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    event: picked-up
    task: 5086 (repo-health-code-pilot, OpenChamber scheduled lane)
    state: claimed
    verified: #3365 is open; no matching open PR found. Nearby open #3443 concerns invalid rollup counts, a distinct field from #3365 timestamp/limit/resetsAt/pair validation. Packet includes a prior real-code probe; I will recheck the current main state before routing.
    next: Verify the reported rollup/record gate mismatch against current main and post triage evidence plus a focused follow-up.

  4. added
    work/assignedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 1, 2026
  5. 67 remaining items

  6. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/claimedQueue worker lifecycle state
    on Nov 2, 2026
  7. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · parked

  8. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  9. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 2, 2026
  10. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/aggregatorPackage/app: aggregator (pm-stack in this repo)app/pm-stackpm-stack (Pad/Grist) in developing-today/codeeffort/sSmall: expected under 2 hours of workkind/bugSomething isn't working (prefixed form of the classic bug label)priority/lowNice to have; do when convenientsource/creatorFiled by the new-issue-creator jobtask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions