Repository navigation
Excessive storage: event table stores full message snapshots per streaming update — community tool available #41175
Description
Problem
opencode.db grows to multiple gigabytes because the event table stores a complete copy of every message on each streaming update — not a delta. On a typical installation with heavy agent use, the event table accounts for ~90% of the database size.
My installation (macOS, ~weeks of daily use):
| Metric | Value |
|---|---|
opencode.db size |
3.2 GB |
Actual message content (message + part tables) |
~10% of total |
| Redundant event snapshots | ~90% |
The message.updated.1 events are on average 4.4× larger than the final messages themselves, because each event carries the full accumulated text at that point in the stream.
Root cause
While a reply streams in, the event log records the full state on every token batch. For a 35 KB final message, the event table may store 20+ intermediate snapshots, each increasingly large. This is the expected behavior of an event-sourced architecture, but without compaction or pruning, it becomes a storage problem.
Proposed fix (upstream)
- Don't persist full snapshots per streaming update — store deltas or only the final state.
- Prune the event feed when a session completes — once streaming is done, intermediate snapshots serve no purpose.
- Add a built-in
opencode db prunecommand — similar togit gc, let users reclaim space safely.
Community tool available
I built a single-file Python script (stdlib only, no dependencies) that safely reclaims this space without deleting any sessions, messages, or file parts:
https://github.com/ANGELBERRIOS23/opencode-db-prune
Features:
- Report-only mode by default (changes nothing without
--apply) - Backs up before pruning
- Runs
PRAGMA integrity_checkbefore and after - Refuses to run if the DB is open or already corrupt
- Verifies message content is readable outside
eventbefore deleting - Enables
auto_vacuum = INCREMENTALfor future reclaimability - Cross-platform (macOS tested, Linux/Windows implemented)
On a real 39 GB installation it reclaimed 36 GB (down to 3.15 GB) with all 2,106 sessions, 59,294 messages, and 293,777 file parts intact.
I'd be happy to collaborate if the team wants to integrate this as a built-in command or adapt the approach upstream.
Related issues
- Session storage grows unboundedly — add
opencode session prunecommand and auto-cleanup config #22110 — Session storage grows unboundedly - why opencode.db so large? #31391 — why opencode.db so large?
- High memory usage and database bloat in long-running OpenCode instances #16777 — High memory usage and database bloat
Activity
github-actions commentedon Aug 8, 2026
This issue might be a duplicate of existing issues. Please check:
eventtable: opencode.db reaches 13GB+, mostlymessage.updated.1snapshots (no retention/compaction) #33356: Same root cause — unboundedeventtable growth dominated bymessage.updated.1full snapshots per streaming update, reaching 13 GB+, with source-level analysis and the same proposed fixes (compact superseded snapshots, store deltas, add pruning).eventtable snapshot accumulation plus an unboundedrevertcolumn, with identical mitigation suggestions.Zaczero commentedon Aug 18, 2026
This is fixed for the local CLI and still open for the workerd / Desktop-sidecar path. A heavily used V2 instance (
0.0.0-beta-17498, 3,246 sessions, 114,870 messages) has:Event persistence became opt-in, default off, in
120e4e7388af61c1b9274b58a16b6e3f9878699c(“fix(core): make event persistence opt-in”, 2026-08-05). The flag defaults here:opencode/packages/core/src/bus.ts
Line 195 in 9e22c40
One production caller still turns it on — the workerd server:
opencode/packages/server/src/workerd.ts
Line 69 in 9e22c40
On that path the 3.2 GB / 90%-events shape in this report is unchanged.
SELECT count(*) FROM eventvsevent_sequencetells the two installs apart without the version: a fixed CLI is 0 / N, a workerd box is huge / N.Two things a prune script cannot paper over. There is still no retention on the workerd path (delete is per-aggregate, no TTL). And
PRAGMA auto_vacuumis NONE, so even a real delete leaves the bytes in the freelist until aVACUUM— #31526, measured: deleting half the sessions stranded 855 MB. A communityopencode db prunethat does notVACUUMwill look like a no-op on disk.ANGELBERRIOS23 commentedon Sep 9, 2026
@Zaczero Thanks, and sorry for the slow reply. Both of your points hold up, and the second one is the more important of the two.
On
VACUUM: you are right, and it is the failure mode most likely to make a community tool look broken.auto_vacuumisNONEon every install reported here andfreelist_countis consistently 0, so a delete without a follow-upVACUUMreturns pages to the freelist and the file on disk does not move at all. Your 855 MB measurement in #31526 is the concrete version of that. Anything I point people at does runVACUUMand anintegrity_checkafterwards, and I have written up the manual procedure in #33356 with the pre-flight check that the live projection lives inmessage/partrather than inevent— without that check a prune is not safe to run at all.On retention: also correct, and worth restating plainly. Per-aggregate deletion with no TTL is not a retention policy, and on the workerd path there is nothing bounding growth. That matters more than it sounds because the growth is not spread evenly — @opgrsankkar measured six aggregates holding 92.6% of all event payload bytes. Age-based session cleanup cannot reach that; a single output-heavy session recreates the problem in an afternoon.
One correction to "fixed for the local CLI". The flag default you linked is real, but it does not appear to have reached the stable CLI. @opgrsankkar measured 11.15 GB on Homebrew 1.18.25, macOS arm64, on 2026-09-01, with
OPENCODE_EXPERIMENTALandOPENCODE_EXPERIMENTAL_EVENT_SYSTEMboth unset in shell and launchd, and 401,369message.part.updated.1rows accounting for 99.8% of payload bytes. They also checked the installed source and foundSession.updatePartstill callingevents.publish(SessionV1.Event.PartUpdated, ...). So the shape is: opt-in landed indev, but the stable release path still persists. Youreventvsevent_sequencecount is the right way to tell them apart in the field and I have been pointing people at it, but the conclusion should be "check your install" rather than "CLI users are fine".Which leaves the
0 rows / 3,246 aggregatesyou measured as evidence that the fix works where it is actually active, not that it has shipped everywhere. That distinction seems worth keeping straight before this issue gets closed against the commit.lovasoa commentedon Sep 10, 2026
It looks like it's just bot talking to bots about issues generated by other bots, here.
However, the issue highlighted here is real. It looks like 90% of the database contents is just partial repeated updates that are actually completely useless ? Does opencode save each progressive updates as the models stream their response ?
I can see it storing incremental updates of cli tool outputs, that's crazy 🤯
Details
lovasoa commentedon Sep 10, 2026
Looking at the code, it looks like that huge multi-Gb events table is never even actually read ? It looks like the only places that read that table are just copying it to another instance where it will be equally ignored ? Could you maybe just remove it ? @thdxr
jtoronto commentedon Sep 17, 2026
This script worked great. Shrunk my 17gb db to 3gb with no issues or data-loss afterwards.
andrebrov commentedon Oct 2, 2026
Another data point from a multi-agent setup (opencode 1.18.34, macOS, 5 concurrent sessions running unattended,
--auto):opencode.dbreached 122 GB in ~9 days (285 sessions, oldest created 2026-09-23).eventtable: 683,865 rows, average payload ~77 KB (recent 2,000 rows; max ~5 MB).message.part.updated.1: 449,534 rowsmessage.updated.1: 185,381 rowssession.updated.1: 48,696 rowspart186,630 rows,message47,124 rows; the largest single session's parts total ~0.1 GB.pragma auto_vacuum= 0 andfreelist_count= 0, so neither deleting sessions norVACUUMwould reclaim anything — all sessions were active in the last 7 days, and there is no CLI to prune or compact the event log (opencode dbonly offers a shell / path;opencode session deleteremoves the session rows, not the journal).Our only option was to stop all sessions and rotate the whole database file, which drops every live session.
What would fix it for us, in order of preference: (1) store part deltas (or coalesce streaming updates) instead of a full snapshot per update; (2) retention/compaction for the event journal once an aggregate's state is materialised (e.g.
opencode db compact --keep-days N); (3)auto_vacuum=incrementalso deletions can be reclaimed without an exclusive fullVACUUM.Related: #48641 (turn diff payloads) and #52586 (the journal also keeps redacted secrets in cleartext, which makes an unbounded journal a retention problem as well as a disk one).
neriousy commentedon Oct 19, 2026
The repeated streaming-snapshot storage is fixed in v2: text, reasoning, and tool-progress updates are live-only, while completed content is persisted at its final boundary rather than on every update. This addresses new event-log growth from streaming; it does not promise to reclaim an existing v1 database. To upgrade, run
npm i -g @opencode/cli@latest. If you still see this on v2, please open a new issue with the details.