Repository navigation
Token display overstates usage for session-backed proxy providers — surface cache/delta or trim resent context #394
Description
Problem
For session-backed providers (the local CLI proxies: commandcode via ACP, muse-proxy, codex-proxy, agy-proxy), OpenCode's per-message token record shows the FULL resent context as fresh input on every turn and reports cache.read = 0, even when the downstream session served almost everything from cache. The display therefore overstates real usage for exactly the providers where the warm-session work (developing-today/code#480) is cutting cost, and any quota/failover decision made from these numbers is made from wrong data.
Evidence (2026-10-24, verified live)
End-to-end verification session ses_e914921a9ffeickeigSWt8x9Pr on the sandbox server, provider commandcode, model deepseek/deepseek-v4.1-flash, two turns against the resident-ACP commandcode proxy:
- Turn 1 assistant message (
msg_16eb6e0c3001Ntlxn2l6w7J4DV): tokens{input: 26103, output: 3, cache: {read: 0, write: 0}} - Turn 2 assistant message (
msg_16eb93165001ZSOEC17Hs5PA5O): tokens{input: 26129, output: 56, cache: {read: 0, write: 0}}
Turn 2 was a delta turn against a persistent cmd session: the proxy sent ~32 chars of new content downstream (vs an 18,899-char full history) and ACP usage reported ~99% of replayed context as cache-read. None of that is visible in OpenCode's record — both turns read as ~26.1k fresh input.
Same shape on the other CLI proxies: the muse route carries ~27k tokens of CLI session scaffolding per call, all displayed as fresh input every turn.
Why it matters
- The user tracks token usage and makes provider/quota decisions from OpenCode's numbers (plan allowances, failover choices in the keeper's registry). Systematically overstating proxy-provider usage skews those decisions.
- As warm-session proxies land (code#480), the gap between displayed and real usage grows — the better the caching works, the more wrong the display looks by comparison (it never improves).
Fix options (either, or both)
- Display/accounting fix: when a provider response carries usage that distinguishes cache reads (or the proxy reports ACP/session usage), ingest it into the message token record (
cache.read) and show net figures in the UI, so a cached turn reads as mostly cache, not fresh input. - Bake the trimming into OpenCode: for providers known to be session-backed (a provider capability flag), OpenCode itself sends delta-only context after the first turn instead of resending the full history, and accounts tokens accordingly — moving the optimization from per-proxy hacks into the client.
Acceptance criteria
- A two-turn conversation against a session-backed proxy provider shows turn 2 with the bulk of its context under cache read (or a materially reduced input figure), matching the proxy's own usage report within a reasonable tolerance.
- Non-proxy providers' token records are unchanged.
- The chosen approach is noted on code#480 so the proxy program and the client stay in agreement about who owns trimming.
Related: #9 (prompt cache optimization across model pools) — adjacent, not the same problem: #9 is about routing for cache hits; this is about the token record/display being wrong for session-backed providers, and optionally moving trimming into the client.
Activity
dezren39 commentedon Oct 26, 2026
Intake review
Fleet monitor intake pass (2026-10-26).
Verified state today — still live
cache.read = 0on a delta turn) has no counter-evidence on main.Related
Risks
lfor that reason.Suggested approach
cache.readwhere a response carries it, so the record matches the proxy's own report. Treat option 2 as a follow-on decision recorded on code#480.Labels applied:
kind/bug,priority/normal,effort/l,status/needs-triage,meta/enriched. The body already carried acceptance criteria, so it was not edited.dezren39 commentedon Oct 28, 2026
🤖 Agent note · state: triaged — confirmed, fix routed
5e77693704, confirmed by code citation (not the issue text alone):Evidence
packages/core/src/session/usage.ts:10-19— the token record is a verbatim map of the AI-layerUsage:input←nonCachedInputTokens,cache.read←cacheReadInputTokens. Nothing downstream corrects it.packages/ai/src/protocols/openai-chat.tsmapUsage(~:915-937) — the cached figure is derived only fromprompt_tokens_details.cached_tokens ?? prompt_cache_hit_tokens ?? cached_tokens ?? cache_read_input_tokens, andpackages/ai/src/protocols/shared.tssubtractTokensreturns the inclusive total unchanged when that figure is undefined. A session-backed proxy whose cache/session usage arrives in any other shape (the issue's ACP usage report) therefore records all input as fresh andcache.read = 0— exactly the 2026-10-24 measurement ({input: 26129, cache: {read: 0}}on a ~32-char delta turn).5e77693704finds no file underpackages/naming commandcode, muse-proxy, codex-proxy, or agy-proxy: there is no session-backed capability flag and no ingestion of a downstream ACP session's usage into the record.Route: a scoped
small_fixpacket for the issue's option 1 (accounting) is enqueued inrepo-health-opencode-pilot(hidden_files/packet-394-fix.json, deduperepo-health:v1:fix:developing-today/opencode:issue:394): first pin the exact usage payload the developing-today/code#480 proxies return, then extend the protocol usage mapping so that figure lands incacheReadInputTokens, with protocol + session-usage tests. Option 2 (client-side delta trimming) is excluded from the packet — it stays a decision to record on code#480, per the issue's acceptance criteria. The issue stays open for the fix.dezren39 commentedon Oct 28, 2026
🤖 Dispatcher hand-back — executor-saturated · openchamber-dispatcher run 2026-10-28 18:00 CDT
small_fix, low risk, no flags, priority 50/normal, timebox 45m) ↔ this issue (Token display overstates usage for session-backed proxy providers — surface cache/delta or trim resent context #394). Packet verified this run: labels clean (kind/bug, priority/normal, queue/queued, effort/l, meta/enriched — no held / user/owned / do-not-merge / do-not-auto-edit, no work/*, no queue/claimed); taskqueuedatt 0 on two independent fleet reads; no open PR for Token display overstates usage for session-backed proxy providers — surface cache/delta or trim resent context #394 (full open-PR list checked viagh); no live session for Token display overstates usage for session-backed proxy providers — surface cache/delta or trim resent context #394 (all 12 session titles/bindings belong to earlier dispatcher tasks)./api/session-activityreads this run (~18:03 and ~18:12 CDT). 0 free slots, so no session was launched for this task./api/quota/*read ~18:03 CDT — meta 5h 1% (reset 9:23 PM) / weekly 21%; opencode-go 5h 32.86% (reset 7:26 PM) / weekly 29.59% / monthly 77.28%; cline-pass 5h 0% in its fresh window (rolled at 6:00 PM, reset 11:01 PM) / weekly 65% / monthly 89%. Saturation is slots only. (OpenRouter monthly remains walled at a known 100%; claude/command-code/antigravity/opendesign reads failed — those pools blocked only themselves.)queued, attempts 0, visible to the owning lane — there is no targeted claim/park verb for a dispatcher hand-back (the known binding gap, fix(script): stats.ts fetches have no deadline and buffer unbounded .json() bodies — PostHog, npm downloads, and the GitHub releases pager #1337/fix(changelog): read upstream changelog through a 1 MiB capped reader #1272), and the generic queue claim would claim a different task, so no park verb was misfired at the wrong task. This comment is the hand-back record with reason classexecutor-saturated; the next dispatcher run with a free slot should dispatch it (Workhorse by default — effort/l implementation work: ingest proxy-reported cache usage into the token record for session-backed providers, packages/ai usage mapping + protocol/session-usage tests, per the packet), or a lane may claim it if urgent.dezren39 commentedon Oct 29, 2026
Queue task 2900 investigation (Lane A) — payload pinned, root cause located outside this repo:
What the proxies actually return (developing-today/code
bin/, read at code main 2026-10-29):commandcode-proxy.mjsreads the ACP prompt result's usage (result._meta.usage/result.usage, ~line 686) intopromptTokens/completionTokens/cacheReadTokens(~line 691). Live-verified figures in its own comments:cacheReadTokens21632/21778 and 19840/19934 (~99% of input). But the OpenAI-shaped usage it emits — streaming end-chunk and non-streaming body alike — contains onlyprompt_tokens,completion_tokens,total_tokens.cacheReadTokensis captured and never emitted.muse-proxy.mjs: itssession/tokenUsagehandler (~line 616) takes onlyinputTokens/outputTokens, though MSP usage carriescacheReadTokenstoo (its own comment: 26225 cache-read of 26455 input). Emitted usage likewise has no cached field.codex-proxy.mjsandagy-proxy.mjs: same shape — no cached field is read or emitted.This repo needs no code change:
packages/ai/src/protocols/openai-chat.tsmapUsagealready derivescacheReadInputTokensfromusage.prompt_tokens_details.cached_tokens(first in its fallback chain) and netsnonCachedInputTokensviasubtractTokens, for both streaming and non-streaming paths;packages/ai/test/provider/openai-chat.test.ts("parses text and usage stream fixtures") already pins exactly that mapping.packages/core/src/session/usage.tsthen recordscache.readverbatim. The moment a proxy emitsprompt_tokens_details: { cached_tokens: N }, turn-2 records showcache.read = Nand only the remainder as fresh input.Set-remainder (tracked here per the cross-repo rule; do not re-file per-repo): developing-today/code proxies must emit
prompt_tokens_details.cached_tokensfrom their session usage — noted on developing-today/code#480. Task 2900 is parked on that landing; it will then verify end-to-end and complete. Option 2 (client-side delta trimming) remains out of scope for this issue and is a separate decision on code#480.15 remaining items
dezren39 commentedon Nov 3, 2026
🤖 Agent note · done — resolved end-to-end; closing
3fe37e8a-c61b-419c-9e97-769ecd264d5d, queue task 7037 (triage)cache.read = 0for session-backed proxy providers.Where the fix actually lives (option 1 — display/accounting)
Proxy side —
developing-today/code(PR anomalyco#2164, merged 2026-10-30T03:45:54Z, merge28a70c2b52; verified at code maina8dbda7fc):bin/commandcode-proxy.mjs(stream end-chunk:1303, non-stream body:1328),bin/muse-proxy.mjs,bin/codex-proxy.mjs, andbin/agy-proxy.mjsnow emitusage.prompt_tokens_details.cached_tokenswhen the session's cache-read figure is > 0.Client side — this repo, already capable and unchanged by this triage (verified at
origin/main3cb0dcfa7e):packages/ai/src/protocols/openai-chat.ts:917-932—mapUsagereadsprompt_tokens_details.cached_tokensintocacheReadInputTokens, and computesnonCachedInputTokens = input − (cached + cacheWrite).packages/core/src/session/usage.ts:11-17—tokens()mapsinput ← nonCachedInputTokens,cache.read ← cacheReadInputTokens.cache.readand shows a materially reducedinput, instead of ~26k fresh input every turn.Acceptance criteria
mapUsagealready handled it for every OpenAI-compatible provider before this change.No further fix task needed. Closing as completed.
developing-today commentedon Nov 3, 2026
agent-note:v1
state: reviewed (coverage loop-back)
Coverage pass 2026-11-03, loop-back review: this issue was re-reviewed (drift / overlap / already-handled / stale / blockers / ordering) so every open issue in the four repos is reviewed and in the system. Verdict: still live; in the system as task 8532 (triage, queued, priority 30) in repo-health-opencode-pilot. Labels audited and synced. If it is already handled at current main, its triage closes it with evidence; if oversized, the triage files the split; blockers belong on this issue.
Found via: coverage loop-back 2026-11-03
dezren39 commentedon Nov 3, 2026
Triage (2026-11-03, queue task 8532) — already closed at 14:11Z; task born stale
Fresh state: #394 is CLOSED (closedAt 2026-11-03T14:11:29Z, stateReason COMPLETED). Nothing is in flight and no fix is needed.
Closure evidence (already on this issue): the closing agent note from queue task 7037 records the resolution end-to-end — the display/accounting gap is fixed at both ends:
developing-today/codePR anomalyco#2164 (merged 2026-10-30T03:45:54Z,28a70c2b52) madecommandcode/muse/codex/agyproxies emitusage.prompt_tokens_details.cached_tokenswhen the session cache-read figure is > 0, and the fork ingests that into the message token record.Why this task existed: the coverage loop-back posted "still live; in the system as task 8532" here at 16:39Z — 2h28m after the close — and this triage was queued from that read, then claimed at 16:54Z.
This is now a pattern in one run (recorded for the fleet monitor). Three of the four tasks this lane claimed in ~15 minutes were loop-back re-queues whose source state had already moved: #1148 (closed 16:08Z, re-queued 16:32Z), #394 (closed 14:11Z, re-queued 16:39Z), and #1146 (live tracker, triaged twice already today, third task queued). Only #1154 carried a real delta. Suggested fixes, cheapest first: (1) re-check issue state at claim and settle with
already_donein one step — what this lane does, so no work is wasted, only one claim each; (2) have the coverage pass read issue state after it finishes collecting, not per-item during the pass; (3) treat a triage verdict newer than ~24h as "in the system".Disposition:
already_done, no follow-up task, no queue or status markers left on the closed issue.