[Bug] Location inactivity eviction SIGTERMs live background shells after 60 minutes #51828

Activity

1. github-actions

   assigned

   rekram1-node

   on Sep 28, 2026
2. github-actions commented on Sep 28, 2026

   

   github-actionsBot

   on Sep 28, 2026 – with GitHub Actions

   Contributor

   More actions

   ---

   This issue might be a duplicate of existing issues. Please check:

   • #48691: Reports terminals killed when a location is evicted after 60 minutes; the same inactivity eviction tears down live shell/PTY processes.
   • #51343: Reports 60-minute location eviction interrupting a still-running session and background work; the underlying location-activity eviction behavior is closely related.
3. SamSpiri commented on Oct 3, 2026

   

   SamSpiri

   on Oct 3, 2026

   More actions

   Second independent instance of background-shell completion loss (v2.0.22)

   Independent confirmation of the exact symptom, with the completion notification proven absent from storage.

   Session ses_efeb61f29ffeWnthm3xB8kevap, background shell job sh_10280cd3f001TXnljvbScsSUtv (launched 18:02:34). Session ended its turn at 18:03:05 (last durable event in the location). Location idle TTL elapsed ~19:03; the job actually finished 19:11:33. No notification was ever delivered; session was woken manually at 20:11.

   • No synthetic row with that shellID anywhere in session_message — if delivery had happened the append-only row would exist.
   • session_inbox / session_pending empty for the session — rules out "admitted but wake dropped".
   • No server restart in the window (NRestarts=0), so restart recovery did not run.
   • .out file removed, matching the watcher's interrupt finalizer (tool/plugin/shell.ts:234).
   • Control: a second background: true job in the same session ran ~17 min and delivered a synthetic ({source:"shell", shellID, jobID, state:"completed", exit:0}) normally.

   Root mechanism: ShellTool.notifyWhenDone is Effect.forkIn(scope, …) into the location/plugin scope (tool/plugin/shell.ts:154-186); the LocationActivity sweep interrupts owners and calls locations.invalidate(ref) (location-activity.ts:51,77-81), closing that scope and interrupting the watcher before sessions.synthetic(...) runs. The job process itself survives (detached), so the command completes with nowhere to report.

   Detail for matching duplicates: the deadline is idle time since the last durable event in the location (1-min sweep), not job runtime — any background: true job expected to outlive ~60 min of location inactivity loses its completion notification.

   (Also posted a longer version on #52908.)
4. TonyYanS commented on Oct 9, 2026

   

   TonyYanS

   on Oct 9, 2026

   More actions

   Independent confirmation on 2.0.24: live shell terminated and automatic summary lost

   Environment

   • OpenCode: 2.0.24, latest channel; source baseline e7a34f09bfd9134dfade5a8ddb843f7030bc9a69.
   • OS: Darwin 27.0.0, arm64.
   • Terminal: Ghostty; shell: /bin/zsh.
   • The installed client has two local UI-only patches (composer visibility and location-less forms); the affected Core lifecycle code was unchanged when observed.
   • Production has two local plugins for continuity metadata and session-history archival. The accelerated reproduction used an isolated profile without those plugins, synthetic data and no external model calls.

   Observed behavior

   A read-only observation command was launched through the agent's shell tool with background: true and timeout: 0, intended to run for 75 minutes. It wrote its own timestamps and an interruption receipt.

   Redacted timeline (UTC, 2026-10-09):

   00:48:47.109  location services evicted directory=<workspace>
   00:48:47.143  observation receipt: interrupted, signal=SIGTERM
   00:48:47.178  GET /api/shell/<shell-id>/output -> 404
   

   The observation ran for 3,700.6 seconds and retained 62 samples. Its on-disk summary survived, but the owning session did not automatically receive a completion summary. The eviction and 404 share the same server run ID; this event was Location eviction, not a server restart.

   Reproduction

   1. Ask the agent to start a harmless long-running shell, e.g. sleep 4500; printf 'BACKGROUND_DONE\n', with background: true and timeout: 0.
   2. Let the agent become idle and avoid further durable Session activity in that Location.
   3. Wait beyond the 60-minute Location idle TTL and sweep interval.
   4. Observe Location eviction, loss of shell observability, and the missing automatic completion notification.

   The threshold is time since the Location's last relevant activity, not a fixed timeout measured from process startup.

   Source analysis and local verification

   In the tested baseline, packages/core/src/location-activity.ts only refreshed activity from durable Session events. The background Shell and ShellTool.notifyWhenDone observer have Location-owned lifetimes. Eviction can tear down both while the owning session is idle.

   An isolated local candidate checks the already-cached Shell service for running commands before idle eviction, refreshes activity on shell creation/exit, and rechecks the activity entry before invalidation. This is supporting evidence, not a claim that either upstream PR has been tested or merged.

   Validation:

   • Accelerated real-child reproduction: 1-second Location TTL, 3-second command. Unpatched success/nonzero cases lost the Location before completion.
   • Patched success, nonzero exit, timeout and explicit cancellation each admitted a notification and woke the original session once; notification markers were acknowledged and ordinary idle eviction resumed afterward.
   • 67 lifecycle/restart/Job/Shell tests passed; Core typecheck and root bun run check passed.
   • Compiled candidate, isolated service and loopback model fixture: background command -> synthetic completion notice -> model wake -> automatic assistant summary passed for exit 0 and exit 7.
   • Six additional macOS bash-related tool-shell tests failed identically on the unchanged baseline; these are not claimed as passing.

   The candidate has not been activated in production, and a full-length production soak of it is still pending. Actual server-restart recovery (resume: false for idle shell owners) is a separate behavior, not addressed by this idle-eviction change.

   Related pending fixes: #52580 and #51827. Please include completion-notification delivery/owner wake, not only child-process survival, in regression coverage.
5. AlexeyShirshov commented on Oct 14, 2026

   

   AlexeyShirshov

   on Oct 14, 2026

   More actions

   Cross-link: #55305 asks for the 60-minute location TTL to be configurable (or opt-out).

   Same root cause as this report — LocationActivity's deadline expires an idle location, and the eviction tears down the location-scoped Shell/Pty graph. If the exemption logic discussed here lands, the knob may still be wanted for hosts that intentionally stay idle for long periods (an agent waiting on a human answer), which is what #55305 covers. Other surfaces of the same deadline: #51343 (running session interrupted), #48691 (terminals killed).
6. luoming319-cloud commented on Oct 23, 2026

   

   luoming319-cloud

   on Oct 23, 2026

   More actions

   Windows data point from the original reporter of #58049/#58050 (Windows_NT 10.0.19045, pwsh).

   Timeline on 2.0.32 (2026-10-22): background shell (background=true, timeout=0) started T0 → location services evicted at ~T+61min → shell process gone, its output file under ~/.local/share/opencode/shell/<hash>/ deleted, GET /api/shell/<id>/output → 404, no completion notification. (Filed as #58049 / #58050.)

   Verification on 2.1.3 (2026-10-23, same box):

   • Background shell: sleep 5400 (90 min), background=true, timeout=0; owning session fully idle for the whole window (well past the 60-min idle TTL).
   • Result: SOAK_BEGIN=2026-10-23T08:56:53+08:00 → SOAK_COMPLETE=2026-10-23T10:26:55+08:00; automatic completion notification delivered to the owning session; output file intact.
   • Service log for the window: no location services evicted entry for the workspace directory, while a co-resident idle location (C:\Users\a, different directory) was evicted on-schedule at its 61-minute mark in the same service run — TTL machinery intact, the keep-with-running-jobs guard doing exactly what the title says.

   Conclusion: #57416 holds on Windows in 2.1.3, and experimental.location_idle_timeout (#57476) is present as the configurability knob.

   Agreeing with the regression-coverage note above: delivery of the completion notification + owner wake should be tested alongside child-process survival.
7. AlexeyShirshov commented on Oct 23, 2026

   

   AlexeyShirshov

   on Oct 23, 2026

   More actions

   Independent verification on Linux (WSL2, 6.18.33.2-microsoft-standard-WSL2), OpenCode 2.1.3. Short version: the guard from #57416 and the knob from #57476 both work here, and the completion-notification gap raised above is now covered by a regression test.

   1. experimental.location_idle_timeout is applied at runtime

   Two isolated opencode serve instances, separate XDG_CONFIG_HOME/XDG_DATA_HOME, no session or model involved:

   • with { "experimental": { "location_idle_timeout": "10 seconds" } }, an idle location produced location services evicted directory=… on the next sweep;
   • a control instance with the default config kept the same location loaded for the whole window (>2 min).

   One detail worth knowing: the setting changes only the deadline, the sweep interval stays at one minute, so the effective precision is one minute, not the configured value. The 2.1.3 binary contains both pieces (Job.runningBackground and the config field), and packages/core/test/location-activity.test.ts passes (5 tests: sessions waiting on a question, plus background shell / subagent child / subagent parent owners).

   2. Notification delivery past the idle TTL is now tested

   This is the coverage requested earlier in this thread. I added a regression case to packages/core/test/tool-shell.test.ts that uses the real shell tool, the real Job service, and the real LocationActivity with a controlled activity clock and 100 ms sweeps:

   1. start a background sleep 3 from the owning session;
   2. jump the activity clock past a full TTL while the session stays silent;
   3. assert the location is still cached while the job runs;
   4. assert the session.inbox.enqueued synthetic completion notification is delivered;
   5. after the job finishes, jump the TTL again and assert the location is evicted.

   Result: 1 pass, 0 fail, bun typecheck in packages/core clean. Step 5 is the control: the same TTL still evicts an unoccupied location, so the keep in step 3 comes from the job guard rather than from a TTL that never fires.

   Two limits of this evidence, to keep the record straight:

   • It covers notification admission/delivery, not a full model-driven long soak. The shared free-model pool is exhausted on this machine ("Free usage limit reached" on every free model, including outside the sandbox), so I did not run an agent-driven 90-minute scenario.
   • The literal POST /api/shell repro in the report is still unprotected. That endpoint calls Shell.create directly with no Session and no Job registration, so neither SessionExecution.active nor Job.runningBackground yields an owner for the location and eviction still terminates the child. The guard covers background shells launched through the tool, which is the path that lost the notifications here.

   Happy to open a PR with the regression test if that is useful — it is currently a local change only.