Repository navigation
Unbounded growth of the event table: opencode.db reaches 13GB+, mostly message.updated.1 snapshots (no retention/compaction) #33356
Description
Summary
The local SQLite store (~/.local/share/opencode/opencode.db) grows unbounded on long-lived instances because the event-sourcing event table is never pruned, capped, or compacted. On two of our long-running instances it reached ~13 GB each, filling a 22 GB volume to 97–99% and eventually producing sqlite3.OperationalError: database or disk is full on ordinary queries — the instance becomes effectively unusable.
The actual conversation data is tiny by comparison; >90% of the DB is the event log, and within it a single event type (message.updated.1) dominates.
Measured breakdown (real instance, 13.7 GB DB)
page_size=4096 page_count=3342166 total=13.69 GB
freelist_count=18 dead=0.00 GB (0.0%) <- live data, VACUUM does not help
Per-table (sum(length(data))):
| table | rows | size | avg/row |
|---|---|---|---|
| event | 141,951 | 12.20 GB | 83.9 KB |
| message | 6,955 | 1.15 GB | 161 KB |
| part | 27,775 | 0.07 GB | 2.4 KB |
| others | — | ~0 | — |
Within event, by type:
| event type | rows | size | avg |
|---|---|---|---|
message.updated.1 |
26,463 | 11.94 GB | 440 KB |
session.next.tool.success.1 |
7,843 | 0.12 GB | 14.8 KB |
message.part.updated.1 |
52,644 | 0.10 GB | 1.8 KB |
| everything else | ~55k | < 0.3 GB | — |
So ~87% of the entire database is message.updated.1 events. With ~6,955 messages there are ~3.8 message.updated events per message, each persisting what looks like a full message snapshot (avg 440 KB), and all historical snapshots are retained forever.
The same pattern was observed independently on a second instance (also ~13 GB, same event-table dominance), so this is reproducible, not a one-off.
Root cause (confirmed in source, dev branch)
packages/core/src/event/sql.ts — the event table has no TTL, no row cap, no size limit, and no compaction column; it just stores every event's full data JSON:
export const EventTable = sqliteTable("event", {
id: text().$type<EventV2.ID>().primaryKey(),
aggregate_id: text().notNull().references(() => EventSequenceTable.aggregate_id, { onDelete: "cascade" }),
seq: integer().notNull(),
type: text().notNull(),
data: text({ mode: "json" }).$type<Record<string, unknown>>().notNull(),
}, ...)packages/core/src/event.ts — the only delete against event is a full-aggregate cascade, used when an entire session/aggregate is removed:
function remove(aggregateID: string) {
return db.transaction(() => Effect.gen(function* () {
yield* db.delete(EventSequenceTable).where(eq(EventSequenceTable.aggregate_id, aggregateID)).run()
yield* db.delete(EventTable).where(eq(EventTable.aggregate_id, aggregateID)).run()
})).pipe(Effect.orDie)
}There is no partial pruning — nothing collapses superseded message.updated.* snapshots, keeps only the latest per message, caps per-aggregate history, or stores deltas. packages/opencode/src/sync/README.md confirms this is an intentional append-only log "for session replayability." That's a reasonable design goal, but with full-snapshot message.updated events and no retention it means every session's event history accumulates for the lifetime of the session, with no upper bound.
SessionCompaction.prune does not address this — it trims context tokens sent to the model, not persisted event rows.
Impact
- Disk exhaustion on long-lived instances (containers/sandboxes/servers with persistent home dirs).
- Once the volume is full, even read queries that need temp space fail with
database or disk is full. VACUUMcannot reclaim it (freelist ≈ 0%; it is all live data). The only current remediation is deleting whole sessions (cascade) or manuallyDELETE FROM event ....
Suggested fixes (any one would help)
- Compact superseded snapshots: for snapshot-style events like
message.updated.*, retain only the latest event per(aggregate_id, message_id)(or per messageseq), deleting older ones — newer fully supersedes older. - Store deltas instead of full snapshots for
message.updated(the README already notessession.updatedwas moved to fields-only;message.updatedappears to still carry a full snapshot). - Per-aggregate retention cap / periodic compaction of the event log for inactive/completed sessions.
Environment
- Observed on builds in the 1.15.x–1.17.x range; root cause confirmed present on
dev(currentlatest, 1.17.9) via the source above. - SQLite store at
~/.local/share/opencode/opencode.db.
Happy to provide the full per-session breakdown or a repro script if useful.
Activity
github-actions commentedon Jun 22, 2026
This issue might be a duplicate of existing issues. Please check:
message.updated.1events accumulating in theeventtable causing the database to grow unboundedly. That issue focuses on OOM when loading old sessions; this one focuses on disk exhaustion at a larger scale (13 GB), but both trace back to the same full-snapshot-per-streaming-update design and lack of compaction/pruning for historical events.auto_vacuumbeing disabled so deleted rows don't reclaim disk space. Different root cause (reclaiming freed pages vs. live data never being deleted), but same overall symptom of unbounded disk growth.If your issue is primarily about the
message.updated.1snapshot accumulation, #32005 is the closest match and combining discussion there may be useful.joaomj commentedon Jul 6, 2026
Same issue here. After a few months of regular use,
~/.local/share/opencodewas consuming over 50 GB split roughly 45 GB in tool output artifacts and 5.7 GB in the SQLite database (opencode.db). There is no built-in mechanism to cap or prune either, and neithercompaction.prunenortool_output.max_lines/bytesaddresses persisted storage, only in-context token limits.As noted in the original issue, the event table grows unbounded with full-snapshot
message.updatedevents that are never compacted. Combined with tool outputs saved to disk for every turn, this makes long-term usage on storage-constrained machines (laptops, containers, VPS) untenable without manual intervention or third-party tools like ocgc.A configurable retention policy (max DB age, max size cap, or auto-compaction of superseded events) would be extremely valuable.
pspranger-throw commentedon Jul 11, 2026
Adding to that:
Reproduction (one command → ~2.3 GB of events)
A single bash tool call downloading a model file:
wget -q --show-progress "https://huggingface.co/<...>/model.Q4_K_M.gguf" -O model.gguf 2>&1
Result for that one part: 79,083 events, 2,325 MB — 97.7% of the session's event cost and ~75% of an 3.1 GB opencode.db. The part's actual final content was 81 KB, so the overhead here was ~29,000×.
Environment: OpenCode 1.17.18, Linux terminal, SQLite WAL mode, auto_vacuum=0.
Where the bytes actually are
For the affected session, the event table was 84% of the DB. Within it:
event type rows total size avg
message.part.updated.1 178,062 2,470 MB 14.5 KB
message.updated.1 36,939 16 MB 456 B
session.updated.1 6,874 3.8 MB 573 B
So in our case the dominant type was message.part.updated.1 (part-level streaming snapshots), ~154× more costly by bytes than message.updated.1. One single part held 2,325 MB of the 2,380 MB total — the bloat is extremely concentrated, not spread across many sessions.
The waste is byte-identical consecutive snapshots
Growth of that one part across its 79,083 updates:
update # part size
1st 322 B
1,000th 31,017 B
10,000th 31,017 B
40,000th 31,017 B
79,083rd (last) 83,352 B
From update ~1,000 to 40,000 the part content was byte-for-byte identical (the progress bar at full size, unchanged).
OpenCode re-snapshotted the same 31 KB 39,000 times — ~1.2 GB of pure duplicates that carry zero information.
Any command that emits thousands of output lines triggers this: wget --show-progress, tail -f, verbose build logs, yes/loops, etc.
EgilSandfeld commentedon Jul 12, 2026
Confirmed on Windows x64 with OpenCode v1.17.18.
opencode.db reached 10.69 GB:
Tool breakdown exposed the write amplification clearly:
PRAGMA freelist_count is 0, so this is live append-only data—not reclaimable SQLite/VACUUM bloat.
Active sessions already account for 2.96 GiB and sessions archived less than one day for another 2.67 GiB. Session retention therefore cannot prevent this; intermediate tool/message updates need deltas or terminal checkpoints instead of repeated full durable snapshots.
chubes4 commentedon Jul 13, 2026
Revised safety model in #36710: compaction is explicit and dry-run by default, limited to 1,000 rows per transaction, and validates the current message/part projection against the retained latest snapshot. The sync protocol has no checkpoint-marker version negotiation, so apply is allowed only for aggregates with no workspace or sync owner; synced/owned aggregates remain report-only pending a protocol compatibility decision.
The supplied production measurement of
message.part.updated.1is 2,614,320 rows / 17.73 GB (within a 20.07 GB event table), which is the primary target for bounded remediation.kitlangton commentedon Jul 19, 2026
The same unbounded-retention shape is present in the current V2
next-15834channel database, although V2's dominant event types differ from the V1 snapshot data in this issue. Read-only aggregate measurements show 2,125 Sessions, 1,302,717 event rows, 3.04 GiB of event JSON, and a 6.09 GiB SQLite file.session.tool.success.1alone occupies 1,833.6 MiB across 168,146 rows, followed bysession.tool.progress.1at 483.8 MiB andsession.reasoning.ended.1at 245.0 MiB. The largest stored event is 27,266,659 bytes.This volume was measured without reading or publishing payload contents, Session IDs, titles, or paths. It is not the direct cause of the continuous multi-TUI CPU incident tracked in #36441, which is driven by live global fan-out, but it independently increases disk use and the cost of cold database reads. V2 retention/compaction should therefore cover large tool/progress events and attachments, not only superseded
message.updatedsnapshots.fix(shell): throttle streaming output metadata updates to cut CPU
ZhiLin-Sam commentedon Jul 22, 2026
Lol same 🙃 Script to keep it under control while we wait: https://gist.github.com/ZhiLin-Sam/552a6454704e31be0164cf9c6cbe70fd
rafaelbn commentedon Jul 23, 2026
I can confirm the same unbounded event-log growth on OpenCode Desktop/CLI 1.18.4 for Windows.
Database diagnostics:
opencode.db: approximately 2.49 GBPRAGMA page_size: 4096PRAGMA page_count: 608158PRAGMA freelist_count: 0PRAGMA auto_vacuum: 0PRAGMA quick_check:okEvent table:
Breakdown:
message.updated.1: 19,588 rows, 2,030,847,015 bytes, maximum row 19,157,539 bytesmessage.part.updated.1: 67,575 rows, 181,888,345 bytessession.updated.1: 5,959 rows, 3,709,489 bytesA single session aggregate contains 28,363 events and 1,891,336,715 bytes, approximately 76% of the complete database.
session.revertandsession.summary_diffsare empty, so the dominant cause here is specifically the retainedmessage.updated.1snapshots.Because
freelist_countis zero,VACUUMalone cannot help. The database is healthy but almost entirely occupied by live event history. This is also creating a significant performance and memory risk for Desktop.I can run additional anonymized SQL diagnostics if useful. Official safe compaction or migration guidance for existing databases would be especially helpful.
chubes4 commentedon Jul 23, 2026
Mine got over 50gb and regrows pretty fast when I clean it up. I think this should be prioritized. As we get into deeper orchestration we need to handle storage carefully. Disk space fills up quick. 1TB does not go very far!
joaomj commentedon Jul 23, 2026
does this happen even if you set
"snapshot":falsein opencode.json?rayborg commentedon Jul 25, 2026
I have a similar project-switch/bootstrap failure with a long-lived but structurally healthy database:
message.part.updated.1andmessage.updated.1snapshots: approximately 99.7% of event JSONPRAGMA quick_check:ok; freelist pages: 0I captured repeated sidecar V8 OOMs on Desktop 1.18.3, 1.18.4, and 1.18.5. The visible trigger is switching projects or bootstrapping the selected project: the UI hangs, V8 reports allocation-failure scavenges followed by
Zone Allocation failed - process out of memory, the sidecar exits with code 5, and renderer requests then fail withTypeError: Failed to fetch.This looks consistent with the unbounded event-snapshot growth reported here and with the OOM behavior in #38362. PR #36710 also appears relevant as proposed bounded compaction work, although it is not currently a supported recovery path for this database.
All diagnostics were read-only and aggregate-only. I did not inspect or include local paths, project/session identifiers, titles, or payload content. No event deletion,
VACUUM, third-party cleanup, or other unsupported cleanup was applied, so the database remains available as evidence.Could maintainers provide supported recovery guidance for an existing database of this size, ideally including a safe dry run and handling for synced/owned aggregates? Guidance on what diagnostic would best confirm the project-switch/bootstrap loading path would also help.
23 remaining items
arul28 commentedon Sep 26, 2026
Independent reproduction with the same shape, plus the exact payload that makes it unbounded — from ADE's use of the pinned
v1.18.32server.Store:
~/.local/share/opencode/opencode.db= 12.5 GB, grew ~160 MB/hour in active use.freelist_count=0, so this is live data, not free pages.dbstatattribution:eventmessagepartBy event type (
SUM(LENGTH(data))):message.updated.1message.part.updated.1session.updated.1Earlier sample from the same machine: 621
eventrows over 1 MB, together 3.68 GB, largest row 88.76 MB.Payload. Every
message.updatedre-publishes the entire message, and for an assistant message that includes the complete file-diff summary:{"sessionID":"ses_...","info":{"id":"msg_...","role":"user","summary":{"diffs":[{"file":"...","patch":"<entire git patch>"}]}}}So one patch is copied into every subsequent snapshot of that message, with no retention, cap, or offload. Only 55 sessions / 5 projects produced 12.5 GB.
Why a workspace hits this: a full-auto chat that touches a few large files accumulates multi-hundred-KB patches per message; a long-running loop (
tool → message.updated → tool → …) multiplies that per turn.packages/core/src/session/message-updater.tsandpackages/core/src/event/sql.tsare the same ondev, so this is not fixed by a version bump.What we're asking for: retention/compaction for the event log and/or moving
summary.diffs[].patchout ofmessage.updatedpayloads (reference it, or bound it). ADE now isolates its own OpenCode data home and ships an opt-in whole-session prune, but that is a workaround for every consumer of this store, not a fix.iceteaSA commentedon Sep 26, 2026
@arul28 #42771 does the second thing you ask for.
summary.diffs[].patchmoves to amessage_difftable,message.updatedkeeps only each file's name, status and line counts, and clients fetch a patch when the diff is expanded. On a 51 GB database, diff-carryingmessage.updatedevents dropped from about 433 KB to 4.4 KB on average (largest 60 MB → 2.3 MB), and it has been running on that machine since Aug 15.@peterradzisz it also adds an opt-in
retention.event_idle_days. Sessions idle longer than that lose their event history through the sameEventV2.removethat deleting a session uses, which removes the session's events and its sequence row together, so it doesn't leave the sequence holes described above. It does nothing for sessions that stay active. With the patches gone, their events are small (about 1.2 KB per row on that machine now), but the table still grows with use.chubes4 commentedon Sep 26, 2026
Can we please do something about this already? Maintaining the OpenCode database is such a massive pain in the ass! I cannot believe nobody from their team has gotten fed up with this unbounded growth yet. Surely they are all using this product heavily???
kidvanguard commentedon Sep 29, 2026
One more data point, plus a policy that actually bounds the table — which is different from what #43456 landed on.
Store:
opencode.db= 5.7 GB on1.18.25.freelist_count = 0andauto_vacuum = 0, so it is live rows;VACUUMalone reclaims nothing (agrees with the measurements above).Redundancy, not just size. The log stores a full snapshot per streamed delta, so the same object is rewritten repeatedly:
message.part.updated.1$.part.id)message.updated.1$.info.id)event.aggregate_idis the session id for 100% of rows (1,434,078/1,434,078), so the log ages with its session — and 59% of the 5.7 GB file (82% of the event table) is events for sessions idle more than a day (2.93 GB for idle >7d, 0.41 GB for 1-7d, 0.32 GB for <1d).What bounds it. Not retiring sessions — that is what #43456 concluded too ("orphan-only pruning reclaims almost nothing... 85 GB of events remained for live sessions"). Deleting the events of sessions that are merely idle does bound it, and the sessions stay readable because the durable state is in
message/part:On the 5.7 GB database that removed 1,374,504 rows / 3.35 GB. After it, the table holds about one day of use (~0.3 GB/day) instead of growing without limit.
session/message/part/todorow counts were identical before and after, andPRAGMA integrity_checkwasok;wal_checkpoint(TRUNCATE)+VACUUMthen took the file from 5.8 GB to 1.8 GB.So a
--max-age-hours-style event policy (drop events for idle sessions, always exempt the newest) would be the property worth having indb prune. Note that #43456 is closed unmerged, and I could not find a merged PR that bounds this table, so it still looks open on 1.18.33.fix(opencode): combine idle-event and session pruning with newest-ses…
AndyS77 commentedon Sep 29, 2026
Your measurements confirm exactly what's needed — deleting events for idle sessions (not the sessions themselves) is what bounds the table, and the durable state in
message/partkeeps everything readable.I've updated #43456 to implement precisely the policy you described.
pruneIdleEvents()does the sameDELETE FROM event WHERE aggregate_id IN (SELECT id FROM session WHERE time_updated < :cutoff AND time_updated IS NOT NULL AND id <> :newest)— always exempting the newest session.What changed since the version you saw:
--max-ageis in hours (e.g.--max-age 168for 7 days), matching the--max-age-hours-style policy you proposed.--purge-sessionsflag for the aggressive mode (also deletes sessions, not just events).VACUUM INTO+ file swap instead of plainVACUUM, to avoid WAL blow-up on large databases. On your 5.7 GB database, plainVACUUMworked becauseauto_vacuum=0and you checkpointed first, butVACUUM INTOis safer in general — it writes a compacted copy to a separate file, then swaps it in after closing the connection, so the WAL never grows by the full database size.Usage:
#43456 was closed by automated PR cleanup (>1 month, <2 reactions). I've requested reopen — the PR now implements exactly the event-pruning policy your data validates. If a maintainer reopens it, this is ready for review.
fix(opencode): use VACUUM INTO + file swap and add --max-age pruning
fix(opencode): combine idle-event and session pruning with newest-ses…
test(opencode): add tests for idle-event pruning edge cases
fix(event): collapse superseded message and part snapshots in the dur…
lfgranja commentedon Oct 3, 2026
Adding measured evidence from a third instance that reproduced this, plus a validated remediation path. Apologies for the length — the detail is what makes it actionable.
Measured instance
opencode.dbbeforepage_count=2345774,freelist_count=0)eventpayloadmessage.updated.1eventConfirmed on 1.18.34, so the "confirmed in source on
dev" analysis in the original report still holds for a released build.New finding: the payload is dominated by
role=user, and by git diffsThis is specific to my instance and I don't see it called out above. Breaking
message.updated.1down byinfo.role:User messages are ~66× larger than assistant messages because
User.summary.diffs[](FileDiff.Info[], seepackages/schema/src/v1/session.ts) embeds full unified git diffs inside every snapshot. The largest two rows in my database were 27 185 810 bytes each — byte-identical content in two different sessions (samemsg_…, same.gitignorepatch), differing only insessionID. A singlemsg_0fa55a79…was stored 196 times (38 MB).Mechanism: any
git addthat touches.gitignore(or similar) inflates a user message, and the streaming loop re-emits a complete snapshot of it per token. So the multiplier isdiffs size × token count.Why
VACUUMcannot help (confirms the original report)All live data.
VACUUMrewrites the file at the same size.A remediation that worked (7.1 GB reclaimed, data intact)
Since the projections are what the UI reads, pruning superseded
message.updated.1snapshots is safe. Three properties frompackages/core/src/event.tsandpackages/core/src/session/projector.tsmake it so:commitDurableEvent()runs the projectors and theinsert(EventTable)in the samedb.transaction(..., { behavior: "immediate" }). There is no window wheremessage/partlagevent.latestSequence()readsevent_sequence, notevent. Verified: for every aggregate,event_sequence.seq == max(seq in event) + 1. Preservingevent_sequencemeans the next event getsseq = 10403, no collision on the uniqueevent_aggregate_seq_idx.MessageUpdatedprojects via upsert —insert(MessageTable).onConflictDoUpdate({ target: MessageTable.id, set: { data } }). Only the latest snapshot of each message survives inmessage, so the other ~3.9 per message are redundant by construction.Result:
Post-prune spot-check: old sessions open and render normally in the TUI.
Caveats, stated plainly:
replay()dies on sequence divergence). Irrelevant for single-device, which is why I checked before doing it.message.updated.1had already reappeared withrowidabove the prune point. This is a mitigation, not a fix.message.part.updated.1(843 MiB here, the largest remaining block) I deliberately did not prune — parts are append-only and an older snapshot can be the final state, so removing it needs the projection verification that fix(core): bound event log compaction #36710 was attempting.rowidrather thanLIMIT, since the index is(aggregate_id, type, seq)and filtering ontypealone is a full scan.Two operational gotchas worth adding to any maintenance path
VACUUMuses$SQLITE_TMPDIR(default/tmp), not the DB's directory. On my box/tmpis a 2 GB tmpfs with ~950 MB free.VACUUMfailed withdatabase or disk is fullafter the backup and prune had already completed, even with 29 GB free on/home. It needs space for the live bytes, not the file size ((page_count - freelist_count) * page_size≈ 2 GB in my case, vs. 8.9 GB if you dimension by file size). Preflight should check the effectiveSQLITE_TMPDIRand abort before touching anything.Cascading deletes need
PRAGMA foreign_keys=ONin the maintenance session (as @ljieyao noted in #47729) — the app itself appears to run with FKs off.Suggestion for the fix
Since
MessageUpdatedalready projects by upsert, a compaction pass that retains only the latestmessage.updated.1per(aggregate_id, message_id)would be safe by the same argument. That is suggestion #1 in the original report and what #36710 attempted. It would also be worth makingeventemit deltas rather than full snapshots formessage.updated— the diff payload insummary.diffs[]is what makes each snapshot expensive, and re-sending an unchanged diff on every token is pure amplification.Happy to provide the pruning script if useful.
one-pyy commentedon Oct 5, 2026
Adding one data point for the diagnostics slice of this. A meaningful part of the bloat in my own database came from
metadata.diagnosticson edit-tool parts/events storing the whole workspace's diagnostics on every edit.I wrote a small cleanup tool, opencode-edit-diagnostics-pruner — https://github.com/one-pyy/opencode-edit-diagnostics-pruner — that removes only those diagnostics entries from historical
partrows and completedmessage.part.updated.1snapshots of a session, after a full SQLite online backup and within a single transaction.On my database it removed ~540 MB of JSON across 327 parts + 327 event copies.
To be clear about scope: this only covers the diagnostics portion. It does not touch
message.updated.1growth and provides no general retention or compaction, so it is not a fix for this issue — just a targeted cleanup for one contributor.mihainadas commentedon Oct 17, 2026
Independent reproduction on OpenCode v1.18.35 (2026-10-17), from an automated builder workload:
opencode.db: 20 GB before cleanupevent: 92,639 rows before cleanupmessage.updated: 11,381 rows / 16.8 GB (about 1.5 MB per row)message.part.updated: about 28,000 rows / 90 MBThis has the same full-snapshot amplification described here: each update while a long assistant message streams persists the whole message again, so storage grows roughly with the square of message length. The part-update events were comparatively small in this sample.
I also confirmed that the current configuration schema has no retention, prune, or TTL setting;
compaction.prunetrims tool output in live context rather than persisted storage; snapshots are for filesystem undo; andopencode runhas no ephemeral/no-persist option. The only supported cleanup available in the CLI was whole-session deletion (opencode session delete <id>).As a local stopgap, 213 sessions older than three days were deleted through the CLI (12 failed and remain retryable), reducing events from 92,639 to 42,348 before an idle-time vacuum. A built-in retention bound and/or delta/supersession behavior for
message.updatedwould remove the need for consumers to implement whole-session cleanup themselves.Only aggregate counts and sizes are included here; no session identifiers, prompts, paths, or payload contents were inspected or published.
heymaaz commentedon Oct 23, 2026
We’re seeing this in production with
@opencode/sdk/workerdv2.1.4, usingOpenCodeWorkerd.createin Cloudflare Durable Objects.Each DO runs short sessions and retains them for debugging. After 7 days / 641 sessions, one DO’s SQLite database reached 76.8 MB:
eventeventindexessession_messagePrompts and tool outputs appear in both the durable events and projected messages. Observed growth is approximately 12.7 MB/day.
We also reproduced retention locally twice on v2.1.4 / Bun 1.4.4, using the recorded OpenAI cassette and
Bus.configured({ persist: true }), matching workerd’s forced persistence setting. After one prompt completed and the session became idle:{ "event": { "rows": 8, "bytes": 983 }, "session_message": { "rows": 2, "bytes": 345 } }The assertion expecting zero retained events fails with
Received: 8. This demonstrates the current retention behavior; automatic cleanup is the requested behavior, not an existing guarantee.