Skip to content

server mode: long-running opencode serve accumulates anonymous JS heap/swap; 26.8GiB cgroup peak, WKFastMalloc/JSJITCode #33213

Closed as not planned

Description

@megamen32

Description

opencode serve appears to retain/fragment a large amount of anonymous JS/Bun heap during long-running server-mode usage. On a Linux production host the process grew to a 26.8 GiB cgroup memory peak and left ~2.86 GiB in swap after ~1.5 days. A service restart immediately dropped the process back to ~250 MiB RSS and memory.swap.current=0.

This looks like the same class of long-running memory growth reported in multiple issues, but this report includes Linux cgroup + smaps_rollup data from the server mode process.

Related existing reports I found while searching before opening this:

Environment

  • OS: Ubuntu 22.04.5 LTS
  • Kernel: 6.8.0-124-generic
  • Architecture: x86_64
  • RAM: ~125 GiB
  • Swap: 50 GiB /swapfile
  • opencode version: 1.17.9
  • Install path: ~/.opencode/bin/opencode
  • Mode: long-running server
  • Command:
/home/roomhacker/.opencode/bin/opencode serve --hostname 127.0.0.1 --port 4095

systemd unit is a simple service with Restart=always; no memory limits were configured before this incident.

Observed behavior

After ~1 day 13 hours of uptime:

opencode.service active since Fri 2026-06-19 22:31:16 MSK
Main PID: 2733850
Tasks: 21
MemoryCurrent: ~3.35 GiB
memory.peak: 26867433472 bytes (~26.8 GiB)
memory.swap.current: 2925260800 bytes (~2.86 GiB)
memory.swap.peak: 3846123520 bytes (~3.85 GiB)

Process-level memory before restart:

PID=2733850
RSS≈1.13 GiB
Swap≈2.86 GiB
VSZ≈105 GiB

/proc/$PID/smaps_rollup:

Rss:             1133880 kB
Pss:             1131378 kB
Pss_Dirty:       1094860 kB
Pss_Anon:        1094860 kB
Private_Dirty:   1094860 kB
Anonymous:       1094860 kB
Swap:            2856700 kB
SwapPss:         2856700 kB

Top mappings by swap/RSS were anonymous JS/Bun/WebKit allocations:

[anon:WKFastMalloc]
[anon:JSJITCode]
[anon:JSStructureHeap]

Examples from /proc/$PID/smaps:

1 GiB anonymous arena: Swap≈511M
1 GiB anonymous arena: Swap≈435M
1 GiB anonymous arena: Swap≈313M
1 GiB anonymous arena: Swap≈150M
many [anon:WKFastMalloc] chunks: 20M-70M swapped each

The binary also showed as deleted from the running process:

/home/roomhacker/.opencode/bin/opencode (deleted)

So the service had likely been updated on disk while the old long-running process kept its heap alive.

Impact

The host had other workload too, but opencode.service alone held several GiB of anonymous heap/swap. During overall memory pressure, swap was nearly exhausted. Restarting only opencode.service freed ~3 GiB of swap immediately.

Before restart:

opencode MemoryCurrent≈3.3G
opencode swap≈2.9G
global swap used≈43G

After restart:

opencode MemoryCurrent≈257M
opencode memory.swap.current=0
global swap used≈40G

Expected behavior

Long-running opencode serve should either:

  1. release old session/request/tool-output state when work is done,
  2. provide a configurable retention/GC policy for server mode,
  3. expose memory diagnostics/heap snapshot hooks, or
  4. document recommended production limits/restart policy.

It should not require periodic restarts to prevent anonymous heap/swap buildup.

Reproduction notes

The server was used heavily through a web/frontend client connected to opencode serve. Workloads included long sessions with many tool calls and large command outputs, including package installation/build commands and repeated shell tool runs.

I do not have a minimal deterministic repro yet, but the symptom is measurable in production and matches other long-running memory reports:

systemctl show opencode.service -p MainPID -p MemoryCurrent -p TasksCurrent
cat /sys/fs/cgroup/system.slice/opencode.service/memory.peak
cat /sys/fs/cgroup/system.slice/opencode.service/memory.swap.current
cat /proc/$PID/smaps_rollup

Workaround used locally

I restarted the service and then added a soft cgroup pressure threshold, not a hard kill limit:

systemctl set-property opencode.service MemoryAccounting=yes MemoryHigh=10G

After restart:

opencode.service: active/running
MemoryCurrent≈237M
MemoryHigh=10737418240
MemoryMax=infinity
memory.swap.current=0
oom=0
oom_kill=0
127.0.0.1:4095 LISTEN

Request

Please do not close this as a generic duplicate without tracking the actual server-mode leak/retention path. The useful next step would be either:

  • guidance for collecting Bun/JSC heap snapshots from the bundled opencode binary,
  • a debug flag to expose heap stats in server mode,
  • or a fix/retention policy for long-running sessions/tool output/state.

Activity

  1. Necmttn commented on Jun 21, 2026

    @Necmttn

    For server mode, memory growth needs daemon-level telemetry rather than after-the-fact host forensics. Emit periodic process metrics: rss, heap used/total, external memory, swap, active sessions, open streams, tool calls served, and uptime. Pair that with a leak guard that snapshots the last N session/tool events before restart, so the next report can tie growth to a workload pattern.


    Generated with ax.

  2. megamen32 commented on Jun 22, 2026

    @megamen32
    Author

    I have a more specific repro now. It is not perfect/minimal yet, but it is much more actionable than the original long-running report.

    Trigger

    From the macOS desktop client:

    1. Connect to a remote Linux opencode serve instance.
    2. Click Add project.
    3. Expand a folder in the project picker / browse tree while looking for a project.

    During that operation the memory growth happens on the remote server-side opencode serve process, not only in the local desktop client.

    Server setup for this run

    Before the test I restarted opencode.service to get a clean baseline and raised the cgroup soft limit from 10 GiB to 40 GiB so the test would not be throttled by MemoryHigh:

    MemoryHigh=42949672960
    MemoryMax=infinity
    memory.swap.current=0
    

    The monitored process was:

    PID=4010366
    /home/roomhacker/.opencode/bin/opencode serve --hostname 127.0.0.1 --port 4095
    

    Result from one short run

    Capture duration: ~295 seconds.

    MemoryCurrent:        845M -> 4129M   +3283M
    RSS:                  754M -> 4048M   +3294M
    smaps Anonymous:      687M -> 3973M   +3286M
    Swap:                   0M ->    0M
    FDs:                    41 -> 190
    TCP :4095 total:       197 -> 490
    TCP ESTABLISHED:        34 -> 52
    TCP CLOSE_WAIT:         24 -> 295
    

    Peaks during the run:

    MemoryCurrent peak: 4585M
    RSS peak:           4515M
    Anonymous peak:     4486M
    TCP total peak:      609
    ESTABLISHED peak:    296
    CLOSE_WAIT peak:     295
    

    The clear burst was here:

    00:22:37 -> 00:22:54  (~17 sec)
    Memory +3674M
    TCP total around 451 -> 466
    ESTABLISHED around 270 -> 280
    FDs 161 -> 193
    

    Fastest observed step:

    +751.6M in 2 seconds (~375.8 MiB/s)
    

    smaps evidence

    After the run:

    Rss:             4144880 kB
    Pss:             4142309 kB
    Private_Dirty:   4068456 kB
    Anonymous:       4068456 kB
    Swap:                  0 kB
    

    Largest mapping observed:

    [anon:WKFastMalloc] RSS≈1239.5M Anon≈1239.5M Size≈1440M
    

    There was also a large anonymous arena:

    1 GiB anonymous mapping, RSS≈48-52M, Size=1024M
    

    Why this looks related

    The repro correlates three things during the desktop remote project picker/folder expansion:

    1. server-side anonymous heap growth in opencode serve,
    2. large WKFastMalloc allocations,
    3. many TCP connections on :4095, especially CLOSE_WAIT.

    So this may not just be generic long-running session retention. It may be triggered by remote desktop project discovery / folder tree expansion and/or the server API/streaming path used by that UI.

    I can try to reduce this further later, but this short run already reproduces several GiB of server-side anonymous heap growth in seconds.

  3. megamen32 commented on Jun 22, 2026

    @megamen32
    Author

    Update: reproduced again on a clean 1.17.9 remote server, and this time it crashed the server process.

    Setup

    I updated a second Linux remote server from 1.17.7 to latest 1.17.9 and restarted the server cleanly before the test.

    Server:

    OS: Ubuntu 22.04.5 LTS
    opencode: 1.17.9
    service: opencode-remote.service
    command: /home/roomhacker/.npm-global/bin/opencode serve --hostname 127.0.0.1 --port 4097
    public access: nginx/basic-auth -> systemd-socket-proxyd 4095 -> 127.0.0.1:4097
    fresh PID before test: 539112
    

    Trigger

    Same UI path as before:

    1. macOS desktop client
    2. connect to the remote Linux server
    3. Add project
    4. expand/browse folders in the project picker

    I started the interaction around 01:32 server local time.

    Result

    The memory spike happened immediately during folder expansion/project browsing, then opencode serve aborted and systemd restarted it.

    Window with the spike:

    01:32:40 -> 01:33:16
    MemoryCurrent peak: 11.6 GiB
    RSS peak:            3.23 GiB
    Anonymous peak:      3.17 GiB
    VmSize peak:        95.8 GiB
    Threads peak:          280
    FDs peak:              389
    TCP :4095 peak:        118
    TCP :4097 peak:        314
    

    Fastest observed growth:

    01:32:47 -> 01:32:50
    +1507 MiB in 3 seconds
    ~502 MiB/s
    

    The large growth correlated with a burst of connections/file picker activity:

    01:32:40 -> 01:32:43
    Memory +1192.7M
    TCP :4095 3 -> 79
    TCP :4097 12 -> 241
    FDs 87 -> 202
    
    01:32:47 -> 01:32:50
    Memory +1507.2M
    TCP :4095 79 -> 79
    TCP :4097 243 -> 243
    RSS 1408.7M -> 1911.1M
    Anonymous 1329.7M -> 1843.4M
    

    Crash log

    Journal:

    2026-06-23T01:33:18+0300 opencode[539112]: memory allocation of 61397856 bytes failed
    2026-06-23T01:33:18+0300 opencode[539112]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    2026-06-23T01:33:19+0300 systemd[1]: opencode-remote.service: Main process exited, code=dumped, status=6/ABRT
    2026-06-23T01:33:19+0300 systemd[1]: opencode-remote.service: Failed with result 'core-dump'.
    2026-06-23T01:33:22+0300 systemd[1]: opencode-remote.service: Scheduled restart job, restart counter is at 1.
    2026-06-23T01:33:22+0300 systemd[1]: Started OpenCode Remote Server.
    

    PID transition from the monitor:

    01:27:20 pid=539112  baseline process
    01:33:20 pid=0       service restarting
    01:33:23 pid=572409  restarted process
    

    After restart the service returned to ~400 MiB and stayed up.

    Why this update matters

    This is not only accumulated memory over a long-running session and it is not fixed in 1.17.9. A clean 1.17.9 server process can be crashed within about a minute by the desktop remote project picker / folder expansion path.

    The interesting signals are still the same:

    • server-side growth, not local-only desktop memory,
    • large anonymous heap growth,
    • huge VmSize,
    • hundreds of FDs/TCP connections during project picker expansion,
    • crash from failed allocation shortly after the spike.
  4. megamen32 commented on Jun 22, 2026

    @megamen32
    Author

    https://youtu.be/8-D9uCZE7F8 Repro video for OpenCode remote server memory leak / crash on 1.17.9.

    Trigger:
    macOS desktop client (1.17.9) -> remote Linux opencode serve (1.17.9) -> Add project -> expand/browse folders in project picker.

    The right-side dashboard shows live server-side cgroup/process metrics for opencode serve: MemoryCurrent, RSS, Anonymous memory, VmSize, threads, FDs, TCP connections, PID history, restarts, and journal.

    Dashboard tool:
    https://github.com/megamen32/service-crash-watch

    In previous measured run on the same setup, a clean 1.17.9 server crashed:
    MemoryCurrent peak 11.6 GiB, VmSize 95.8 GiB, +1507 MiB in 3 seconds, then memory allocation failed and status=6/ABRT.

  5. aryansinghrajput3738-code commented on Jun 27, 2026

    @aryansinghrajput3738-code

    http://localhost:3000 npm install

    npm install

    npm start

    node server.js

  6. github-actions commented on Aug 27, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions