Repository navigation
OpenCode 2: V2GenerateReplay keeps a full clone of each session's served history with no cap #764
Description
Magic Context: 0.49.19 (OpenCode 2.1.1, macOS).
What
V2GenerateReplay.capture() (packages/plugin/src/v2/hooks/generate.ts, called from the context hook in v2/hooks/context.ts) stores, per session, clonePreservingInstances copies of the full draft.messages, system, tools and the anchor on every pass. The entry is replaced each pass but only removed by forget() on session.deleted.
Unlike the other per-session caches (LkgEntryDigestCache 16 sessions / 256 MB, NativeFoldCache 16 MB × 16 sessions, RestoredRowCache 16 sessions), this map has no session or byte cap.
Impact (estimated, not measured)
Long sessions here serve about 6,000 messages per request; their stored rows are 89–394 MB each in OpenCode's DB, and the LKG cache's own text estimate for two of them is 23–32 MB. A deep clone of each served history is plausibly tens to hundreds of MB per session, held until that session is deleted or the server restarts, plus a large clone allocation on every pass. Every session touched since the server started keeps an entry.
The OpenCode server process here grew to ~2.4 GB footprint over 25 h (peak 3.0 GB), almost all JS heap. I haven't attributed that growth; this cache is the main unbounded holder I found in Magic Context.
Suggestions
- Cap by sessions (LRU) and/or bytes, like the other caches.
- Or keep only the anchor and model/agent, and rebuild the replay from the LKG slot when
/btwactually needs it. - Avoid the per-pass deep clone when the served prefix hasn't changed.
- Exposing the cache size through
debug_rpcstats would make this measurable.
Related: #666 (same code, cache misses).
Activity
magic-alfonso commentedon Oct 22, 2026
Thanks, @null-axiom. Confirmed:
V2GenerateReplaykeeps an uncapped per-session map, andcapture()deep-clones the full served history on every pass.A fix is in progress. It caps the cache like the other per-session caches and stops cloning the whole history on every pass, while keeping replays byte-identical. We'll post here when it ships.
magic-alfonso commentedon Oct 23, 2026
Fixed in 0.49.20. Thanks for the report.
OpenCode 2 now caps the saved request copies it keeps for side questions: at most 16 sessions, 128 MiB per session and 256 MiB in total, with the least recently used session evicted first. When an evicted session asks again, it sends OpenCode's own request unchanged, never stale bytes. In a 40-session measurement the cache went from 466 MiB and growing to 137 MiB.
null-axiom commentedon Oct 23, 2026
Thanks — confirmed on 0.49.20 that the caps are in place (16 sessions, 128 MiB per session, 256 MiB total, LRU).
One remaining cost:
capture()still deep-clonessystem,messagesandtoolson every pass (v2/hooks/generate.ts, called from the context hook), so each turn allocates a full copy of the served request even when the prefix hasn't changed. That's allocation churn proportional to history size rather than unbounded growth. Skipping the clone when the anchor/prefix is unchanged, or keeping only the anchor and rebuilding on/btw, would remove it.magic-alfonso commentedon Oct 23, 2026
Thanks for confirming the caps.
The per-pass copy is deliberate. The saved request is what a later side question replays, and copying the served
system,messagesandtoolson each pass is what guarantees that replay is byte-for-byte what was sent, even if a host or another plugin mutates those objects afterwards. When we measured it, the copy shares the message text rather than duplicating it and cost about 1.5 ms per pass at 6,000 messages, so we kept it. The retention was the real problem, and that's what 0.49.20 capped.If you see that copy show up as a measurable cost in your sessions, please open a new issue with the numbers and we'll look again.