Skip to content

Memory Megathread #20695

Closed
Closed

Description

@thdxr

We have scattered reports of memory issues so we are going to tackle them all in a central place.

PLEASE DO NOT RUN YOUR LLM AND SUGGEST SOLUTIONS IT IS ALWAYS WRONG

What we need help with is collecting heap snapshots. There are two ways to do that.

Manual snapshot flow

  • run opencode (terminal not desktop)
  • use it until memory has grown (we don't need crazy numbers, once it hits 1-2gb it should help)
  • press ctrl+p and select heap snapshot
  • ui will freeze while it is generated
  • upload generated files here
  • leave a comment describing your issue a bit

Automatic snapshot flow

  • run opencode with OPENCODE_AUTO_HEAP_SNAPSHOT=1, for example OPENCODE_AUTO_HEAP_SNAPSHOT=1 opencode
  • use it until memory grows past around 2gb
  • opencode checks memory once a minute and automatically writes a heap snapshot when rss is above 2gb
  • snapshots are written to the log directory as .heapsnapshot files
  • upload generated files here
  • leave a comment describing your issue a bit and mention that you used the automatic flow

If you follow this we can almost always trace the issue and fix it

Activity

  1. self-assigned this
    on Apr 2, 2026
  2. pinned this issue on Apr 2, 2026
  3. added
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    perfIndicates a performance issue or need for optimization
    on Apr 2, 2026
  4. sjawhar commented on Apr 2, 2026

    @sjawhar

    Hooray, will contribute what I can. Curious to learn if you have any tips for avoiding the false diagnoses from LLMs. Why do they struggle so much, and is it fixable?

  5. heimoshuiyu commented on Apr 2, 2026

    @heimoshuiyu
    Contributor

    How to "write heap snapshot" in web mode?

  6. jpcarranza94 commented on Apr 2, 2026

    @jpcarranza94
    Contributor

    Hey @thdxr , the particular problem I am running into is not while running opencode with the terminal (#17047 in this issue), but rather, when running it in server mode. Any suggested method to help you collect evidence for memory leaks while running it on server mode?

    Also, I think the error that I am mentioning here appears to have a clear one liner fix that avoids RangeErrors for tool calls on a long running server. (I am currently running a fork that serves thousands of requests per day, and without this fix it crashes with RangeErrors on any tool call after 2 hours of running): #16952

    I debugged extensively and added additional logging to tools to be able to understand exactly what was causing the stack overflow type error. They all came from this exact line:

    const toolInfo = init instanceof Function ? await init(initCtx) : init which reuses the same tool definition object for object defined tools, so each init() call wraps the previously wrapped execute and accumulates wrapper closures over time.

    Happy to help provide more logs/heap snapshots.

  7. thdxr commented on Apr 3, 2026

    @thdxr
    MemberAuthor

    @jpcarranza94 great find - merging that

  8. thdxr commented on Apr 3, 2026

    @thdxr
    MemberAuthor

    How to "write heap snapshot" in web mode?

    next release will have a flag for auto snapshots - added stuff in instructions

  9. Sheraff commented on Apr 3, 2026

    @Sheraff

    I'm running the client with opencode attach and the "heap snapshot" command writes the snapshots to undefined and find / -type f -name "server.heapsnapshot" cannot find it either.

    Image

    I'm not really sure my issue would have shown up in the snapshot anyway, because it is not so much a memory leak inside opencode itself, but mainly the fact that it starts a playwright MCP server per unique directory/worktree, and at a cost of 98M per instance it can quickly add up.

    Image

    Maybe some MCPs could be configured to be global?

  10. CarloWood commented on Apr 3, 2026

    @CarloWood
    Contributor

    If anyone can help me to reproduce this (any leak) on my Arch/linux box, then I can probably help finding where that comes from.

  11. sjawhar commented on Apr 4, 2026

    @sjawhar

    Heap snapshots + native memory profiling data

    Snapshots

    I have 8 .heapsnapshot files captured from 3 different processes at various RSS levels (2.5GB to 11GB). Will upload to the Dropbox link.

    • serve-early-359MB.heapsnapshot — opencode serve at ~2.5GB RSS
    • serve-peak-1.6GB.heapsnapshot — same serve at ~11GB RSS
    • tui-299MB.heapsnapshot — TUI session (different process, same leak)

    What we found

    JS heap (5-8GB): FileDiff.before/after fields from snapshot.diffFull() storing entire file contents as strings. 22 copies of a 20MB JSON file = 440MB from one file alone. summary_diffs column in SessionTable loads these into memory on session resume.

    Native memory (140GB+ beyond JS heap): This is the bigger issue. Using memleak-bpfcc with bun-linux-x64-profile for symbol resolution, we captured the allocation stack:

    StringImpl::create8BitIfPossible → AtomStringImpl::add
      → stringProtoFuncSplitFast (String.prototype.split)
        → asyncGeneratorBodyCall → runInternalMicrotask
          → VM::drainMicrotasks → JSNextTickQueue::drain
            → event loop tick
    

    11.5GB in 55 allocations (~210MB each) via bmalloc's pas_large_heap_try_allocate_user_allocation that never get freed. This is the known bmalloc slab retention bug (oven-sh/bun#28318).

    Scenario

    • opencode serve --port 13381 handling 100 sessions across 10 project directories (Legion worker fleet)
    • Also reproduces in TUI sessions with heavy subagent usage
    • RSS grows from 400MB to 150GB+ over 3-4 hours
    • Growth is bursty: 2-10GB spikes in 5-second intervals
    • Fresh session in same directory does NOT leak — resuming existing sessions triggers immediate spike

    Data in Dropbox upload

    • 3 heap snapshots (Chrome DevTools format)
    • heap-analysis.txt — programmatic analysis of snapshot 1 (top types by retained size)
    • heap-diff.txt — diff between snapshot 1 and 2 showing what grew
    • memleak-bpfcc traces with fully resolved native stacks
    • bpftrace mmap syscall traces
    • Memory telemetry timeline (RSS/heap/smaps every 5 seconds)

    Automatic snapshot flow

    Used OPENCODE_AUTO_HEAP_SNAPSHOT=1 on the serve. TUI snapshots captured via kill -USR1.

    Uploading files now.

  12. sjawhar commented on Apr 4, 2026

    @sjawhar

    Heap snapshots and profiling data uploaded here (zstd-compressed, decompress with zstd -d <file>.zst):

    https://github.com/sjawhar/opencode/releases/tag/memory-leak-data

    Contents:

    • serve-early-359MB.heapsnapshot.zst (71MB) — serve at ~2.5GB RSS
    • serve-peak-1.6GB.heapsnapshot.zst (172MB) — same serve at ~11GB RSS
    • tui-299MB.heapsnapshot.zst (113MB) — TUI session
    • heap-analysis.txt — programmatic type/retained-size breakdown
    • heap-diff.txt — what grew between the two serve snapshots
    • memleak-20260404-030902.log — memleak-bpfcc native allocation traces
    • bpftrace-mmap-20260404-030856.log — mmap syscall traces
    • memory-telemetry-stderr.log — RSS/heap/smaps timeline
  13. 140 remaining items

  14. freaksdotcom commented on Aug 27, 2026

    @freaksdotcom

    I captured paired TUI and server heap snapshots from OpenCode 1.18.23 on Linux x86_64. The terminal client was running under tmux for about seven hours with concurrent subagent work during the session.

    Before capture, the OpenCode process used 2.67 GiB RSS and about 84% CPU. /proc reported approximately 2.1 GiB of private anonymous memory before snapshot generation. The main thread used about 29% CPU, seven HeapHelper threads used roughly 6.8% each, and one worker used about 6%. This looked like sustained garbage collection rather than file-backed database memory.

    The paired snapshots differ substantially:

    Snapshot File size Nodes Edges
    TUI 707,391,772 bytes 10,857,279 35,670,487
    Server 149,527,598 bytes 2,407,682 7,374,399

    The TUI heap contains 6,380,387 objects, 3,196,177 closures, and 3,306,738 arrays. One minified closure named sJ has 969,251 live instances and 93,194,866 bytes of self size. Direct incoming references include 215,639 object properties named fn, about 153,000 lexical-context references each under two minified names, and 60,364 array elements. Other direct references use fields found in message and part state, including value, type, time, metadata, input, output, state, text, tool, and sessionID.

    The server heap does not contain the same sJ population. Its largest groups are generic objects, functions, lexical environments, arrays, and maps. This paired result localizes the dominant live-object population to the TUI rather than an MCP child or file-backed SQLite pages. I cannot map sJ to an unminified source symbol without the release build's source map.

    The resumed session had 11,199 persisted event rows containing about 113 MiB of serialized event payload, 404 message rows, and 2,161 part rows. Restarting OpenCode reduced RSS from 2.67 GiB to about 846 MiB and removed the sustained HeapHelper load. The resumed event-heavy session still had a high baseline, but the restart reclaimed about 1.8 GiB.

    I have both raw snapshots and can submit them through the upload channel linked in this issue. Their SHA-256 hashes are:

    2d1c5ae650e605afc143f7bdfc4737cef78e01795b3ea0afefd01b4e93995e99  tui.heapsnapshot
    2c6176d847a7c7dd1f04e8a068e4f1246cbff45f7f099ad19e2f614e6ae0cbe9  server.heapsnapshot
    
  15. NoahBPeterson commented on Sep 4, 2026

    @NoahBPeterson

    @thdxr This is still an issue. Two opencode instances with <200k tokens uses 800 megabytes of memory each, whereas claude code needs <400 megabytes of memory.

    Your dropbox links are down, too.

    Image Image
  16. jeremyakers commented on Sep 5, 2026

    @jeremyakers

    Opencode just crashed with this:

    addListener: [Function: addListener],                                                                                                                                                       
      on: [Function: addListener],                                                                                                                                                                
      prependListener: [Function: prependListener],                                                                                                                                               
      once: [Function: once],                                                                                                                                                                     
      prependOnceListener: [Function: prependOnceListener],                                                                                                                                       
      removeListener: [Function: removeListener],                                                                                                                                                 
      off: [Function: removeListener],                                                                                                                                                            
      removeAllListeners: [Function: removeAllListeners],                                                                                                                                         
      listeners: [Function: listeners],                                                                                                                                                           
      rawListeners: [Function: rawListeners],                                                                                                                                                     
      listenerCount: [Function: listenerCount],                                                                                                                                                   
      eventNames: [Function: eventNames],                                                                                                                                                         
    },~  ⊙ 5 MCP /status                                                                                                                                                                 1.18.29  
        type: "event",                                                                                                                                                                            
       count: 11,
    
          at overflowWarning (node:events:185:19)
          at addListener (node:events:158:22)
          at ~effect/Effect/evaluate (/$bunfs/root/chunk-pz7rtzm0.js:25:4492)
          at runLoop (/$bunfs/root/chunk-pz7rtzm0.js:25:2045)
          at evaluate (/$bunfs/root/chunk-pz7rtzm0.js:25:1435)
          at runTasks (/$bunfs/root/chunk-pz7rtzm0.js:11:12750)
          at afterScheduled (/$bunfs/root/chunk-pz7rtzm0.js:11:12637)
    
    [codegraph-bootstrap] skipped /tmp/opencode: bootstrap already running
    ============================================================
    Bun v1.3.14 (0d9b296a) Linux x64 (baseline)
    Linux Kernel v5.15.0 | glibc v2.35
    CPU: sse42 popcnt
    Args: "opencode" "--user-agent=opencode/1.18.29" "--use-system-ca" "--" "--mdns"
    Features: Bun.stderr(4) Bun.stdin(4) Bun.stdout(4) abort_signal(4) fetch(3828) http_server jsc no_avx2 no_avx spawn(1047296) standalone_executable transpiler_cache(3) tsconfig(2) workers_spawned napi_module_register process_dlopen 
    Builtins: "bun:ffi" "bun:main" "bun:sqlite" "node:assert" "node:async_hooks" "node:buffer" "node:child_process" "node:console" "node:crypto" "node:dns" "node:events" "node:fs" "node:fs/promises" "node:http" "node:https" "node:module" "node:net" "node:os" "node:path" "node:process" "node:querystring" "node:readline" "node:stream" "node:stream/consumers" "node:stream/promises" "node:string_decoder" "node:timers/promises" "node:tty" "node:url" "node:util" "node:zlib" "node:worker_threads" "ws" "node:v8" "node:http2" "node:diagnostics_channel" "node:dgram" 
    Elapsed: 18258321ms | User: 20250600ms | Sys: 3878070ms
    RSS: 12.00GB | Peak: 0.08TB | Commit: 12.00GB | Faults: 1228010 | Machine: 0.10TB
    
    CPU lacks AVX support. Please consider upgrading to a newer CPU.
    panic(main thread): Out of memory while copying request body
    oh no: Bun has crashed. This indicates a bug in Bun, not your code.
    
    To send a redacted crash report to Bun's team,
    please file a GitHub issue using the link below:
    
     https://bun.report/1.3.14/Ba10d9b296mzGuhooigDwujhsC+5qzwCuo7twCu+npwCgkzppDksiusC00hhsC8u5gsCA0eNrzLy1RyE9TyE3NzS+qVCjPyMxJVUjOL6jMzEtXKEotLE0tLlFIyk+pBAAxnw8w
    
    Illegal instruction
    
    
  17. max-flizpay commented on Sep 8, 2026

    @max-flizpay

    What a joke, its been 7 months and we are still nowhere.
    Opencode has lost my trust

  18. VihangaFTW commented on Sep 8, 2026

    @VihangaFTW
  19. thdxr commented on Sep 8, 2026

    @thdxr
    MemberAuthor

    the majority of memory issues have been fixed in opencode2 which is coming out in the following 2 weeks. you can use it now here: http://opencode.ai/v2

  20. unpinned this issue on Sep 8, 2026
  21. thdxr commented on Sep 8, 2026

    @thdxr
    MemberAuthor

    closing this issue and we'll track v2 memory issues as individual issues

  22. Sewer56 commented on Sep 26, 2026

    @Sewer56
    Contributor

    My first experience of V2 is 80+GB RAM usage and OOM 😅
    Which is how I got here.

    On the other hand, V2 is great, especially with how plugin-extensible it is.

  23. Faks commented on Sep 29, 2026

    @Faks

    Hi, seems your fix did not fix memory leak...

    v2.0.18 TUI: 115.6 GB RSS after ~8.5h, session content only 1.1 MB — system-wide OOM

    Debian 13, X11, local llama.cpp model (27B), agent=build, 12–14 plugins. TUI launched in tmux pane 2026-09-28 23:58, OOM-killed 2026-09-29 08:55. Kernel: task=opencode, total-vm: 152025376kB, anon-rss: 121119400kB. Cgroup peak 116G
    RAM / 4.9G swap; total CPU 12m32s (~2% avg). Session: 195 messages, ~142k input / 142k output tokens, ~1.1 MB stored content (largest message 70 KB). The global OOM also killed the KDE Plasma shell via systemd OOMPolicy=stop.
    Happened on the latest release, so it persists in v2.

  24. Sporknife commented on Sep 29, 2026

    @Sporknife

    opencode2 has a bug that when you resize the tui, it will glitch out and cause a memory leaks to fill 16GB of ram (+16Gb of swap) in 20 seconds . its really bad because its caused by the server and not the tui somehow.

  25. simonklee commented on Sep 30, 2026

    @simonklee
    Member

    opencode2 has a bug that when you resize the tui, it will glitch out and cause a memory leaks to fill 16GB of ram (+16Gb of swap) in 20 seconds . its really bad because its caused by the server and not the tui somehow.

    A fix for this is in our v2 branch and will go out next release

  26. Sporknife commented on Sep 30, 2026

    @Sporknife

    @simonklee thank you.

  27. Abdur-rahmaanJ commented on Oct 12, 2026

    @Abdur-rahmaanJ

    CLOSED???

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions