[Bug]: Backend crash leaves Claude agents running after their runs are marked cancelled #15357

Activity

1. juliusmarminge commented on Oct 3, 2026

   

   juliusmarminge

   on Oct 3, 2026

   Member

   More actions

   Note

   Grok responding on behalf of Julius.

   Triage

   Thanks for the precise report and the timeline, @allisonmahmood! The code on current main (5e35272f) matches what you found. A kill -9 of the backend skips every finalizer, and startup recovery then marks the run cancelled while the claude process is still alive.

   What happens

   ClaudeAgentSdkQueryRunner.open calls query() without a spawnClaudeCodeProcess (apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.ts). The child shares the backend's process group and session and gets no parent-death signal. On Linux, Effect spawns the desktop backend detached, so the group id is the backend's pid, and kill -9 of that pid doesn't signal the group. The claude process gets reparented (to systemd --user in your trace) and keeps going.

   Graceful shutdown does close the query. closeSession calls query.close() through a session-scope finalizer, but SIGKILL never runs it. Merged #14926 only made the fake test CLI exit when stdin closes. The real CLI keeps working after that, which is what your /tmp/acted row shows.

   On the next boot, ProviderRuntimeRecoveryService assumes the processes are gone (the comment around line 570), writes run.updated with status: "cancelled", and idles the provider thread. No pid is recorded and nothing is reaped first. The invalid_mcp_credential 401 is the new server correctly refusing the orphan's MCP reconnect. Local tools don't go through that check, so git, file edits, and gh carry on with nothing written back to the thread.

   The idle Monitor case is the same orphan. #15358 is the opposite situation, where the process really is gone and the thread is never told, and #15359 is only about the log line. Open #15323 handles SIGINT and SIGTERM but doesn't record or kill a process that outlived SIGKILL. Upstream, claude-agent-sdk-typescript#142 asked for parent-death termination and was closed as a duplicate.

   Likely fix area

   • On Linux, one option is a wrapper that sets PR_SET_PDEATHSIG and then execs, installed through spawnClaudeCodeProcess. It doesn't carry over to grandchildren, and macOS has no prctl.
   • Another option works across platforms and already exists in the tree as OpenCodeServerLedger. Record the pid, start time, and command, then on the next start kill only a process that still matches, before the cancelled run.updated events are committed. A group kill would need the same start-time check, because the dead backend's pid can be recycled.

   Until then, killing leftover claude processes whose parent is pid 1 or the systemd user manager is the right workaround. A maintainer will decide on the fix direction.
2. juliusmarminge

   added

   bugSomething is broken or behaving incorrectly.Something is broken or behaving incorrectly.

   via-triageFiled through npx t3 triageFiled through npx t3 triage

   on Oct 3, 2026
3. smagnetize commented on Oct 11, 2026

   

   smagnetize

   on Oct 11, 2026

   More actions

   Another reproduction from daily use. It adds one consequence the report doesn't cover: after recovery, restart continuation starts a second agent in the orphan's worktree, so two agents work one checkout.

   For example, on 2026-10-06 the backend was SIGKILLed at 21:41:08 UTC. Recovery marked the runs cancelled, and continuation sent "Continue where you left off" to a fresh Claude session on each thread. Meanwhile the original claude processes, reparented to the systemd user manager, kept working in the same worktrees.

   What two agents in one checkout did

   • A paid call twice. On one thread, the new session pressed the same paid video-generation button 33 seconds after the orphan did, which billed a second clip.
   • Interleaved commits. Both sessions committed on one branch. One commit picked up the other's uncommitted edit.
   • Duplicate GitHub issues. Each session filed the same follow-up issue.
   • The orphan held the real work. On another thread, the orphan kept committing and opened a pull request after T3's run had ended. T3 showed that thread as failed, and a person's retry got "The provider could not start this turn". Stopping the orphan would have lost its uncommitted work, and waking the thread would have added a second agent.
   • The new session couldn't see it. In every case, T3's timeline showed only the new session.

   Environment and frequency

   • Version: t3code-nightly-bin 0.0.46_nightly.20261006.2735, Orchestrator V2, desktop on Arch Linux (kernel 7.2.5), Claude Code via the Agent SDK.
   • desktop.trace.ndjson logs "Process interrupted due to receipt of signal: 'SIGKILL'" for the backend at 2026-10-06 21:41:08, 2026-10-07 19:51:49, 2026-10-07 21:07:23 and 2026-10-08 04:40:24 UTC.
   • We don't know what sent the SIGKILLs. We didn't send them by hand.

   For #15450
   The ledger reap would prevent this, because it runs before recovery and so before continuation. Two suggestions:

   • Keep continuation strictly after the reap.
   • Note in the thread when a previous session was stopped. The orphan can leave uncommitted changes in the worktree that the continued session should know about.

   Our workaround until then: a pre-tool hook refuses edits, commits, pushes and paid calls while another claude or codex process has the same working directory, and names that process so whoever coordinates can pick one.
4. SunkenInTime

   added a commit that references this issue on Oct 12, 2026

   fix(server): agents stop when the server that launched them dies on L…

   ...

   This issue will close once commit f6c095a is merged into the 'main' branch.

   f6c095a
5. ShriD5

   added 2 commits that reference this issue on Oct 15, 2026

   fix(harness): record agent processes so the next engine can end orphans

   ...

   b548482

   feat(engine): reap agents an earlier engine left running at startup

   ...

   758fd20
6. never102 commented on Oct 17, 2026

   

   never102

   on Oct 17, 2026

   More actions

   Another reproduction, on macOS, triggered by an in-app update rather than a crash. PR #15450 covers Linux only (its notes say macOS wasn't run live), so this may help scope the macOS half.

   What happened (T3 Code Nightly, macOS, Claude provider, 2026-10-16)

   • 15:00:43Z: a thread's Claude session opens (query.open with sessionId) and starts a long turn.
   • About 15:02: an in-app update restarts T3 mid-turn.
   • 15:02:54Z: the restarted backend opens the same thread again (query.open with resume set to the same session id) and offers "Continue where you left off." The log has no query.close for the first query.
   • The first claude process (--session-id <id>) is still alive, now parented by launchd (ppid 1). The new one runs --resume <id> under the new backend. Both write to the same session transcript.
   • The orphan kept working for 40 minutes, unseen in the thread: it renamed a git branch in a worktree, rewrote files and launched new background work while the resumed session did the same job in parallel. It stopped only when killed by hand.

   So on macOS the update path does what this issue describes for a SIGKILL on Linux, including the second-agent effect @smagnetize reported.

   Workaround we run meanwhile: a Claude Code SessionStart hook that stops any claude process whose command line carries the same session id and whose parent is pid 1, skipping its own ancestors. It's the triage's second option (a pid ledger) done from the agent side, and it would be redundant once the server reaps its own agents on every platform.
7. SunkenInTime

   added a commit that references this issue on Oct 18, 2026

   fix(server): agents stop when the server that launched them dies on L…

   ...

   This issue will close once commit 637952c is merged into the 'main' branch.

   637952c
8. Cosmonaut-itc commented on Oct 19, 2026

   

   Cosmonaut-itc

   on Oct 19, 2026

   More actions

   Another macOS reproduction from an in-app update, with one detail that may help scope the fix: the update path's graceful stop ends in SIGKILL when the backend is busy, so it lands in the same case as a crash.

   Timeline (T3 Code Nightly, macOS, updating to 0.1.1-nightly.20261018.3335; Claude Code 2.1.303; times from desktop.trace.ndjson, UTC)

   • 04:32:40: desktop.updates.installDownloadedUpdate → desktop.backendInstance.stop.
   • 04:32:46: desktop.backendInstance.finalizeRun. That is about 6 s later, just past DEFAULT_BACKEND_TERMINATE_GRACE (5 s, then forceKillAfter). About seven Claude threads were open and several were mid-turn, so the backend most likely didn't finish closing its queries and was force-killed.
   • 04:32:50: the app relaunches. At 04:33:00 the new backend runs claude --resume=<id> for each thread at once.

   Result

   Three threads ended up with two live claude processes on the same session id. The old ones had ppid 1, the new ones the new backend. Both appended to the same transcript .jsonl. The orphans kept working for 25 to 40 minutes, invisible in T3:

   • One launched a Codex worker that the resumed session later believed it had launched itself.
   • One launched a second review round and stopped a worker that was editing the same worktree.
   • One held a test suite on a remote machine.

   Nothing was lost only because we noticed and had each resumed session read the transcript, take over the orphan's pending work, and kill it.

   Workaround we run meanwhile

   This is similar to the one described above. A Claude Code SessionStart hook finds any other claude process with --input-format stream-json and the same --session-id/--resume value that isn't one of its own ancestors. It then injects its PID and child processes into the new session's context, so the session hands the work over before killing the orphan. It doesn't kill anything itself, because killing the orphan outright would drop in-flight work.

   A server-side fix probably needs both halves: reap or reattach surviving agents before resuming a session, and give the update path's stop enough time, or a clean kill of the provider children, so it never escalates to SIGKILL with agents still attached.
9. Cosmonaut-itc

   added a commit that references this issue on Oct 19, 2026

   feat(claude): hook que avisa de otro proceso en la misma conversación (…

   ...

   4dfdabb
10. coygeek commented on Oct 20, 2026

    

    coygeek

    on Oct 20, 2026

    More actions

    Current-source update to the cause analysis above: the Claude launch and startup cleanup have changed since the original report, but macOS still bypasses the leftover-agent cleanup.

    I inspected the latest published nightly, v0.1.1-nightly.20261020.3411 at d36b7da2485bc889b320a285deea181f637d760a, and refreshed main at 6ab39ff6b8f60447c2580b818351fe4387fffa82. The relevant Claude spawn, agent-scope, and desktop shutdown files are byte-identical between those revisions.

    • The default POSIX Claude launch now uses spawnClaudeProcessGroup, which sets detached: true. The query runner selects this function outside Windows at lines 921–924. Its child-exit callback kills the remainder of the group, but that callback runs in the owning server and cannot run after that server dies. This updates the original diagnosis that Claude shares the backend's process group.
    • AgentScope.make returns the no-op default on every non-Linux platform. The stopLeftoverAgents startup reap in that file therefore does not run on macOS. Its own comment states the intended invariant: stop agents left by a killed server before resumed sessions work in the same checkout.
    • The desktop update path explicitly requests backend stops before quitAndInstall. Normal quit/relaunch also waits for shutdown in apps/desktop/src/app/DesktopLifecycle.ts. The backend still has a five-second termination grace in apps/desktop/src/backend/DesktopBackendManager.ts. These calls establish intended cleanup; they do not establish that the update incidents above timed out or identify their actual termination signal.

    As a bounded mechanism check, I extracted the current spawn function, removed only its TypeScript signature annotations, and ran it with a harmless Node heartbeat provider that continues after stdin EOF. Once per baseline, killing only the captured stand-in server PID with SIGKILL left the provider alive under PPID 1 with advancing heartbeats. It overlapped a second manually launched stand-in provider. In the control, SIGTERM triggered the stand-in server's AbortController and its provider exited. All fixture processes were stopped afterward.

    This was a process-level stub check, not a full T3 Code/real-Claude reproduction. It does not establish real Claude's EOF behavior, automatic restart continuation, or the cause of a particular app restart. The useful source distinction is that current process-group cleanup is parent-owned, while startup reclamation remains Linux-only. The existing macOS incident evidence in this issue is still needed to connect that mechanism to real provider activity.
11. t3dotgg commented on Oct 21, 2026

    

    t3dotgg

    on Oct 21, 2026

    Member

    More actions

    Note

    🤖 Claude Opus 5.5 responding on behalf of Theo

    Thanks for the report. One part of this is fixed on main in #17662. Please test on the nightly built from main on 2026-10-21, or on the next stable release. One part is still open: reap orphaned Claude agents on macOS before recovery or resume. If that part still matters to you, please open a new issue for that part only.
12. t3dotgg

    closed this as completedon Oct 21, 2026