Skip to content

Per-(account,model) exhaustion — 1005 on one model locks the whole account #9

Open

Description

@longphamdev

Problem

Quota buckets are per-model (glm-5.3 daily 3M ≠ glm-5.3-flash campaign 300M — separate grants, live-verified). But AccountPool.markExhausted marks the whole account exhausted on any biz-1005, regardless of which model the request targeted.

Consequence: if glm-5.3's daily bucket is drained but glm-5.3-flash still has 300M, a single glm-5.3 request hitting 1005 locks the account out of Flash traffic too — Flash requests that would succeed get rotated away (or blocked if all accounts exhaust this way). False exhaustion of working capacity.

Current behavior

  • Request glm-5.3 → 1005 → markExhausted(accountId) → entire account skipped until quotaExpiresAt or half-open
  • The 1005 body does NOT carry the model — classification only sees {code:1005,msg:'exceed quota limit'}

Detection (from live logs)

The 1005's logid/request row includes the requested model (printRow shows glm-5.3 vs glm-5.3-flash). The metrics dashboard (#8) counts 1005s per (acc, model) — if we see glm-5.3 1005s coexisting with healthy flash traffic on the same account, this issue is confirmed real in production.

Fix design (when confirmed)

  1. markExhausted(accountId, opts?: { model?: string }) — store per-model exhaustion set alongside account status
  2. pickable() filters by the REQUESTED model: an account with an exhausted glm-5.3 bucket stays pickable for glm-5.3-flash requests
  3. Classification: attach the request's model to the exhaustion (handler knows meta.model at bookkeeping time)
  4. /quota surfaces per-model exhausted status alongside per-model remainingUnits

Scope note

This changes pickable()'s contract (currently model-agnostic). Requires the pool's reserve() modelHint (already wired for strategy) to feed the filter. Touches the failover loop's tried-set semantics (tried per (acc, model)).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions