Skip to content

aggregator: quota gatherer snapshot dedup ignores credentialId — two ok credentials of one provider interleave into one series and append an alternating noise point every tick #1624

Closed

Description

@dezren39

Summary

The quota gatherer's snapshot dedup (recordsFromEntries, pm-stack/aggregator/src/quota-gather-map.ts:243-274) keys its "value last appended" state by serviceId|window only — but the gatherer produces one entry per credential (scripts/gather-quota.ts: fetchQuotaForProviderInstances → entryFromResult per result; serviceIdFor maps by provider, quota-gather-map.ts:63-78), and SnapshotRecord has no credential dimension (src/model.ts:1256-1267). When two credentials of one provider are both ok in a tick, their readings interleave into a single series and the dedup — whose stated purpose is "the store gains a point when a reading moves, not every 15 minutes" (quota-gather-map.ts:228-242) — appends a noise point every tick forever, even when neither value ever changes.

Mechanics, all in recordsFromEntries + the caller's merge (gather-quota.ts:322-340):

  • The dedup key is ${entry.serviceId}|${w.name} (quota-gather-map.ts:254); entry.credentialId is carefully carried onto the entry (:188) and then never used.
  • The comparison reads the input lastAppended map (:256), never the values appended earlier in the same tick, so tick 1 appends both credentials' points for the same series at the same ts.
  • The state map afterwards holds whichever credential was processed last. Next tick, the other credential's (unchanged) value differs from the stored stamp and appends; the merge flips the stored stamp; the following tick the first credential appends again — alternating 10, 20, 10, 20… at the 15-minute tick cadence, indefinitely.

Verified (this run, on main 9752a47)

Probe ON THE LAPTOP (Bun, driving the real recordsFromEntries from quota-gather-map.ts, sha256 b18d3fd9… identical in both clones; state merged between ticks exactly as gather-quota.ts:339 does; probe + log in the goal's hidden_files/probe-2026-10-28-laneB-gatherdedup/):

  • Two ok entries, same serviceId anthropic:claude-pro, window 5h, credential A at 10% and credential B at 20%, values never changing:
    • tick 1 → 2 records appended, same series, same ts: …|5h=10 and …|5h=20
    • tick 2 → 1 record (=10), tick 3 → 1 record (=20), tick 4 → 1 record (=10) — one noise point per tick, alternating, forever.
  • Control, single credential at a constant 10%: tick 1 → 1 record, ticks 2-3 → 0 records; a genuine move 10→15 appends exactly 1. The dedup works precisely as documented for the single-credential case.

Reachability: per-credential results are the gatherer's normal shape — gather-quota.ts reads OpenCode's credential store and fetches per credential (its own comments describe multiple credential rows per provider, e.g. Codex's token-sharing login row). Any provider with two ok credentials (two stored Claude/Codex logins) triggers it; error/unconfigured rows are skipped by the state !== "ok" guard, so a refused second credential does not.

Why it matters

The snapshot store is the hub's long-term usage series (§4): it feeds history, trend and cost views, and its writers are deliberately append-on-change so the series stays sparse and meaningful. In the multi-credential case the series instead gains a fabricated sawtooth — the line oscillates between two accounts' real readings at one point per 15-minute tick — which reads as usage swinging up and down, corrupts any per-service history derived from it, and grows the store ~96 points/day/window for data that never moved. It also makes the two accounts' readings indistinguishable: both are recorded under one serviceId with no credential attribution.

Distinct from #1540 (SnapshotStore append-time validation of non-finite/ non-ISO records — different layer, the store's gate; these records are individually valid) and from #1560 (registry merge forgetting fields).

Acceptance criteria

  • With two ok credentials of one provider in a tick and unchanged values across ticks, the gatherer appends no per-tick noise: either the dedup/series key distinguishes credentials (and the record/series shape documents that), or the mapper's per-provider series rule is made explicit and only one deterministic reading per serviceId+window is recorded per tick — pinned by a test.
  • Tick 1 with two ok credentials no longer appends two contradictory points for the same series at the same timestamp (or the two points are credential-distinguishable, per the pinned rule).
  • Regression tests cover: two-credential constant values over ≥3 ticks (no alternating appends), the single-credential constant case (append once), and a genuine value move (append once).
  • Existing aggregator test suites still pass.

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    task/triageQueue packet kind: verify, reproduce, and route this item
    source/creatorFiled by the new-issue-creator job
    priority/normalOrdinary priority (the default)
    on Oct 28, 2026
  2. developing-today commented on Oct 28, 2026

    @developing-today
    Contributor
  3. added
    queue/queuedHas a queued task in the Supabase agent queue
    effort/sSmall: expected under 2 hours of work
    app/aggregatorPackage/app: aggregator (pm-stack in this repo)
    on Oct 28, 2026
  4. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · filed

    • Agent: new-issue-creator job (lane B, new-issue-creator-code-b)
    • Session / ID: new-issue-creator-code-b cron run, 2026-10-28T08:39Z
    • Found via: filed this run
    • Task: Filed this issue after verifying with a Bun probe on the laptop driving the real recordsFromEntries (two ok credentials, constant values: tick 1 appends both points, ticks 2-4 append one alternating noise point each; single-credential control dedups correctly and a genuine move appends once)
    • Expectation: effort/s — triage pins the series rule (credential-distinguished keys vs one deterministic reading per serviceId+window), then a small fix + regression tests
    • Links: probe + log in the goal's hidden_files/probe-2026-10-28-laneB-gatherdedup/
  5. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Oct 28, 2026
  6. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  7. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · triage findings (queue task 1741)

    • Agent: queue worker (repo-worker-code), Lane D
    • Session / ID: queue task 1741, repo-queue-worker-shift-d, 2026-10-28T13:20Z
    • Found via: queue claim
    • Task: Pin the snapshot series rule for the gatherer dedup defect in this issue.

    Verdict — bug confirmed as filed; series rule PINNED

    The snapshot series stays pool-level: (serviceId, window). Per tick, exactly one deterministic reading per key is recorded: the maximum pctOfLimit across that tick's ok credentials for that provider+window. No SnapshotRecord shape change, no credential dimension in the store.

    Why not credential-distinguished series

    Verified on current main d39e6111 (file contents re-read this run):

    • Every snapshot writer emits pool/service-level records with no credential dimension: recordsFromPools, recordsFromSaasServices, recordsFromProviderUsage (src/snapshot-records.ts). The gatherer is the only writer that ever sees credentials.
    • The store and consumers are keyed that way end to end: SnapshotStore.byService() groups by serviceId only (src/snapshots.ts:82), UsageView.history is Record<serviceId, SnapshotRecord[]> (src/model.ts:1282), and snapshotKey is (serviceId|metricId|ts|value) (src/snapshots.ts:44).
    • serviceIdFor's own contract (src/quota-gather-map.ts:52-57) is that gatherer records continue the series the dated-read writers started. Credential-keyed gatherer records would fork that series for every multi-credential pool and require plumbing a new dimension through validation (isSnapshotRecord), the store key, grouping, and the pace-projection/render consumers — a cross-cutting shape change to fix a dedup bug.
    • Per-credential detail is not lost: it already lives in quota-current.json, where entries carry credentialId/credentialLabel (entryFromResult, quota-gather-map.ts:188-189). Snapshots are pool history; quota-current is per-credential present.

    Why max, specifically

    • Order-independent. Fetch results arrive per credential with no guaranteed order; any "first entry wins" rule can flip when fetch order flips, recreating the sawtooth. Max is the same value regardless of entry order.
    • Conservative for the consumers that exist. The history feeds pace projection / trend views for a pool; the tightest credential is the binding constraint and max never understates it.
    • Append-on-change semantics preserved. With constant per-credential values, the max is constant → zero appends after tick 1. When the max credential's value moves, exactly one append. Documented consequence: a move by a non-max credential that does not change the max appends nothing (that is the pinned pool-level meaning of the series, not a bug).

    Implementation spec (follow-up task enqueued from this triage)

    In recordsFromEntries (src/quota-gather-map.ts:243-274):

    1. Group the tick's ok entries' pct windows by serviceId|window; select max pctOfLimit per key (tie-break: lowest credentialId lexicographically, so the emitted record's ts/resetsAt are deterministic too).
    2. Compare the selected stamp against lastAppended AND track selections in the returned appended map within the same call — the current code only reads the input map (line 256), which is the same-tick half of the bug. At most one record per key per call.
    3. No change needed in scripts/gather-quota.ts: its state merge (line 339) and its snapshotKey fresh-filter both behave once appended/records carry one entry per key (note: snapshotKey includes value, so it never deduped the two contradictory tick-1 points — records-level selection is the only correct fix point).
    4. Document the pinned rule in the recordsFromEntries doc comment, replacing/augmenting the §"value changed" wording at lines 228-242.

    Regression tests (test/quota-gather-map.test.ts), replicating the caller's state merge between ticks:

    • two ok credentials, constant 10/20 over ≥3 ticks → tick 1 appends exactly 1 record (value 20); ticks 2+ append 0;
    • same entries in reversed order → identical tick-1 record;
    • single-credential constant → 1 append, then 0;
    • max credential moves 20→25 → exactly 1 append; non-max move 10→15 (max unchanged) → 0 appends;
    • existing aggregator suites still pass.

    Out of scope (unchanged from the packet): backfilling/cleaning already-interleaved points in existing stores; provider fetching / credential reading.

  8. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

  9. added
    queue/queuedHas a queued task in the Supabase agent queue
    and removed
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    on Oct 28, 2026
  10. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    Queue task 1873 progress on branch queue/1873-1624-snapshot-max-reading — PR: #1775

    What changed (scope: quota-gather-map.ts recordsFromEntries + tests only):

    • recordsFromEntries now groups same-tick entries by serviceId|window and emits one record per key: max pctOfLimit wins, lowest-credentialId tie-break (then earliest measuredAt/resetsAt), deterministic emission order.
    • Dedup check reads both the input state map and this call's appended map, so one call can never emit two records for one key.
    • Pinned rule added to the doc comment: a non-max credential moving while the max is unchanged appends nothing (pool-level semantics).

    Test results:

    • bun test pm-stack/aggregator/test/quota-gather-map.test.ts: 18 pass, 0 fail. New tests replicate the caller state merge between ticks: two ok creds constant 10/20 over 4 ticks -> [1,0,0,0] with tick-1 value 20; order reversal identical; single-cred 1,0,0 control; max 20->25 appends once; non-max 10->15 appends zero; per-call key uniqueness.
    • Wider aggregator suite: 325 pass, 1 fail in actions.test.ts (attention badges) — verified pre-existing on the base commit via stash, unrelated to this fix.
  11. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    🤖 Dispatcher binding · openchamber-dispatcher run 2026-10-28 13:45 CDT (burn-by-expiry run: Drew directive 13:44 CDT — fill free session slots, burn the Go window resetting 2:26 PM and the Meta window resetting 4:23 PM)

  12. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · bound

    • Agent: fleet monitor (repo-fleet-monitor)
    • Session / ID: queue task 1873 · fleet-monitor run 2026-10-29 00:19 CDT
    • Found via: monitor reconciliation (Duty 1B PR↔queue)
    • Task: Binding — open PR fix(aggregator): quota snapshot dedup pinned to pool-level max #1775 implements this issue (verified by the monitor this run: PR title/body/files read; scope matches the queued task's packet). Pickup route: dispatcher session (a "Dispatcher binding" comment was posted on this issue 2026-10-28T18:56Z and checkpoint comments name queue task 1873 and the PR; branch queue/1873-1624-snapshot-max-reading names the task), but the pickup never landed in the queue — task 1873 still sits queued, attempts 0, never claimed. The queued task is therefore to be treated as PARKED on this PR: resume as the PR's shepherd — review CI, merge when green, do NOT re-implement. Queue-side claim/park by the monitor is pending the by-id claim/bind verb (Dispatcher v1 live: scheduled launches of OpenCode/OpenChamber sessions for queued issues #1337); until it lands, this note is the binding record a lane's revalidation will find.
    • Expectation: PR shepherded to merge; no duplicate implementation.
    • Links: PR fix(aggregator): quota snapshot dedup pinned to pool-level max #1775 · queue task 1873
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/aggregatorPackage/app: aggregator (pm-stack in this repo)effort/sSmall: expected under 2 hours of workkind/bugSomething isn't working (prefixed form of the classic bug label)priority/normalOrdinary priority (the default)queue/queuedHas a queued task in the Supabase agent queuesource/creatorFiled by the new-issue-creator jobstatus:in-reviewtask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions