Repository navigation
Plugin worker per session holds 0.96-8.98 GB of VmSwap forever after its RSS collapses (7 processes = 17.9 GB swap, host thrashes) #767
Description
Environment
- Magic Context:
@cortexkit/opencode-magic-context@latest(installed via npm, 2026-10-23) - OpenCode: v2.1.3
- Host: Linux 7.2.11-arch1-1, 32 GB RAM, 31.3 GB swap (zram 15.3 G + swapfile 16 G)
- Session count at time of observation: 8 concurrent sessions, plugin processes aged 15–25 h
What
One plugin worker process per session grew to a VmHWM (peak RSS) of 1.6–2.9 GB, was swapped out by the kernel, and then shrank to an RSS of 10–33 MB while still holding 0.96–8.98 GB of VmSwap with only 8 file descriptors open (idle). The swap slot is never released. The process is effectively an empty shell pinned in swap.
Seven such processes were observed simultaneously, together holding ~17.9 GB of swap, which filled the machine's entire 31.3 GB swap and caused global swap thrashing (si sustained at 1–4 MB/s, wa at 3–15%), making the desktop unusable until the processes were killed.
Measured evidence
Per-process /proc/<pid>/status at the time of triage (before cleanup):
| PID | VmRSS at rest | VmHWM (peak RSS) | VmSwap held | open fds | age |
|---|---|---|---|---|---|
| 347777 | 608 MB | ~1.8 GB | 8,977 MB | 8 | 5 h |
| 7225 | 31 MB | 2,929 MB | 2,686 MB | 8 | 24.7 h |
| 207898 | 11 MB | 1,664 MB | 1,488 MB | 8 | 15.4 h |
| 8806 | 33 MB | 1,676 MB | 1,408 MB | 8 | 24.6 h |
| 198407 | 10 MB | 1,972 MB | 1,334 MB | 8 | 15.8 h |
| 93327 | 11 MB | 1,705 MB | 1,023 MB | 8 | 21.9 h |
| 226133 | 10 MB | 1,593 MB | 962 MB | 8 | 14.9 h |
Common shape of every leaking process:
- peak RSS 1.6–2.9 GB, all of it anonymous JS heap (no large mapped files, no shared memory of that size)
- at rest: RSS collapsed to 10–33 MB, 8 fds only → the process is idle, not working
- yet
VmSwapstays at 0.96–8.98 GB and is never reclaimed - all 7 are children of the same
opencode serve --serviceparent, one per session
Aggregate: ps RSS of the whole opencode process tree was 5.5 GB (13 processes) at the time, while free showed Swap 30 GB used / 31.3 GB total.
Reproduction
No deterministic repro yet. Observed twice, 3 days apart, on the same host:
- Open ~8 sessions, leave the server running for 15+ h with mixed activity (long sessions, subagent dispatches,
ctx_searchcalls). - Watch
VmSwapper plugin process climb monotonically; RSS peaks between 1.6 and 2.9 GB. - Once swap is exhausted the machine thrashes — a mouse move or window switch blocks for hundreds of ms.
killon the leaking plugin processes returns ~17.9 GB of swap immediately; the session itself survives (the parent respawns workers as needed).
Related observations that may share a root cause — see #764 (V2GenerateReplay keeps an uncapped per-session deep clone), #763 (queued pending_ops from finished sessions never reclaimed), #745 (deleted-session cleanup depends on one event loop). This report may be the OS-visible symptom of the same class of problem: per-session state that is allocated per pass but only freed on session.deleted.
What I need from you
- Confirm whether the 1.6–2.9 GB peak is expected working set, or a leak (e.g. per-session caches with no cap, cloned histories retained across passes).
- If it is expected working set: is there a documented way to bound it — a config knob, a byte cap, or a way to force a worker to release its heap without deleting the session?
- A way to observe it from inside the plugin (a
/ctx-status-style dump of per-session retained bytes) would make this diagnosable without reading/proc.
Workaround in use
Killing the leaked plugin processes by PID, plus reducing the host's swap (zram 16 G → 4 G, swapfile 16 G → 8 G) so that a leak can no longer wedge the machine. The workaround does not fix the leak.
Happy to run any instrumented build, attach heap snapshots, or add per-pass logging if you tell me where to look.
Activity
magic-alfonso commentedon Oct 23, 2026
Thanks for the measurements. Pages that stay in swap while the process sits idle are memory it still holds and never touches again, so we're treating this as retention on our side, not normal kernel behaviour. We're reproducing a long OpenCode 2 session to find which per-session structures hold that memory and which ones need a cap. One cache already gets a cap in the next release (from issue 764), but your numbers look larger than that alone.
Two things would help:
ps -o pid,args -p <pid>), so we know exactly which process this is./proc/<pid>/smaps_rollupfor one affected process, if one comes back.We'll post what we find here.
magic-alfonso commentedon Oct 23, 2026
An update from our investigation. On OpenCode 2, our server plugin isn't a process per session: it's loaded into OpenCode's service process and shared by sessions in the same folder. On real OpenCode 2.1.3 with 20,000 messages, its JavaScript heap stayed around 210 MB after garbage collection, with no swap.
The children you're seeing are most likely Magic Context's local embedding processes. With a local embedding model, we start one long-lived process per project. It holds the model's native memory, never shuts down when idle, and on OpenCode 2 isn't released when the project's context is disposed. That matches a few children of
opencode serve --servicethat each grew large once and then sat idle.We're fixing that now. An idle embedding process will shut down after 10 minutes and restart on the next request, and it will be released when its project is no longer in use. To confirm it's the same thing on your machine, could you share the output of
ps -o pid,ppid,rss,args -p <pid>for one of the swapped children, and/proc/<pid>/smaps_rollup? If the command line namesembedding-worker-process.js, that confirms it.magic-alfonso commentedon Oct 25, 2026
Shipped in 0.49.21. Local embedding processes now shut down after 10 minutes without work and start again on the next request. On OpenCode 2, a project's embedding process is also released when the last context using it closes. A request in progress is never cut off. We measured this on the compiled build with the default model: the process was gone at the ten-minute mark, and its replacement produced identical vectors.
We haven't confirmed these were the processes you saw, so this stays open for now. After updating, if idle children still hold memory or swap, please post the command line of one (
ps -o pid,rss,command -p <pid>) and we'll keep looking. Thanks, @LouisLau-art.