Resumed subagent loses completed status after app-server restart #34220

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   app-serverIssues involving app server protocol or interfacesIssues involving app server protocol or interfaces

   subagentIssues involving subagents or multi-agent featuresIssues involving subagents or multi-agent features

   sessionIssues involving session (thread) management, resuming, forking, naming, archivingIssues involving session (thread) management, resuming, forking, naming, archiving

   on Jul 19, 2026
2. harjotgill commented on Jul 19, 2026

   

   harjotgill

   on Jul 19, 2026

   Author

   More actions

   I implemented and validated the proposed fix in commit 5daeddc5a on harjotgill:agent/restore-resumed-agent-status.

   The patch is ready for maintainer review, but GitHub currently rejects both CLI and GitHub App attempts to open the cross-repository PR because external PR creation is not permitted for this account. A maintainer invitation or an internal cherry-pick is required by the repository's contribution policy.

   🎉React with 🎉1Reacted by Ignat Remizov
3. github-actions commented on Jul 19, 2026

   

   github-actionsbot

   on Jul 19, 2026 – with GitHub Actions

   Contributor

   More actions

   Potential duplicates detected. Please review them and close your issue if it is a duplicate.

   • Parent session waits indefinitely after resume because subagents remain paused #34043

   Powered by Codex Action
4. ignatremizov commented on Jul 20, 2026

   

   ignatremizov

   on Jul 20, 2026

   More actions

   This sounds useful after a crash
5. bbingz commented on Jul 25, 2026

   

   bbingz

   on Jul 25, 2026

   More actions

   Analysis (community)

   Residual analysis for #34220 (resumed subagent loses Completed status after restart) under docs/contributing.md — invitation-only. No unsolicited PR.

   Root-cause hypothesis

   Your trace is exactly right, and it can be pinned to three lines: the AgentStatus watch channel is initialized unconditionally at PendingInit inside spawn_internal (codex-rs/core/src/session/mod.rs:718) — a constructor every session type traverses, including InitialHistory::Resumed — and the only writers are live-event paths (mod.rs:1914 terminal-turn notify, mod.rs:2086 deliver_event_raw via agent_status_from_event). Nothing in the resume branch folds the rollout's terminal events back into the channel, so a child that completed before shutdown is observed as PendingInit forever. Verified at HEAD 4c43465133:

   1. The status derivation already exists — it's just never applied on resume. agent_status_from_event (core/src/agent/status.rs:6-21) maps TurnComplete → Completed(last_agent_message), TurnAborted → Interrupted/Errored, Error → Errored, ShutdownComplete → Shutdown; is_final (:23-28) excludes PendingInit/Running/Interrupted. Reconstructing status on resume is therefore a fold of an existing pure function over the trailing rollout events — no new state machine.

   2. Your ShutdownComplete caveat is structurally correct, not just defensive. Because is_final treats Shutdown as final, a naive "replay the last status-bearing event" would let a trailing ShutdownComplete (the prior process's lifecycle marker) overwrite the preceding TurnComplete, erasing Completed(Some(final_message)) — the exact failure your reproduction guards against. The fold must prefer the last turn-outcome event (TurnComplete/TurnAborted/Error) over a trailing ShutdownComplete. Your branch 5daeddc5a gets this right.

   3. The wait_agent timeout is downstream, not a second bug. The completion watcher loops while !is_final(&status) (core/src/agent/control.rs:457, with the post-loop re-check at :468), so a resumed child stuck at PendingInit is indistinguishable from a child that never started — wait_agent blocks the full timeout even though the final message is sitting in the rollout. Seeding the channel on resume fixes the wait path with zero changes to those loops.

   4. One boundary the fix must respect (flagging because it wasn't spelled out in the issue): the seeding must apply only to a cold InitialHistory::Resumed. When the app-server reconnects to a session that is still live in another process, the reconstructed history can end mid-turn; folding that to a terminal status would wrongly mark a running agent Interrupted. A cheap discriminator: a trailing unfinished TurnStarted (no matching TurnComplete/TurnAborted) maps to Interrupted only when the rollout's owning process is gone — which is precisely your "unfinished persisted TurnStarted should resume as Interrupted" proposal, stated as a liveness condition.

   Fix outline (all OSS, codex-rs): in the resume path after reconstruct_history_from_rollout/replace_history, add a helper in agent/status.rs that folds agent_status_from_event over the trailing rollout events with the turn-outcome-over-ShutdownComplete precedence above, and send_replace the result into agent_status_tx. Fail-first restart coverage: (1) child completes → clean manager shutdown → resume root → agent_status() of the resumed child is Completed(Some(msg)) and wait_agent returns immediately with the final message (today: PendingInit, blocks to timeout); (2) rollout ending TurnComplete then ShutdownComplete still yields Completed, not Shutdown; (3) trailing unfinished TurnStarted from a dead process yields Interrupted, not Running. Happy to draft the test patch if a maintainer wants it — though reporter harjotgill's validated branch 5daeddc5a already implements the core, and per the invitation-only policy the realistic landing path is a maintainer cherry-pick.

   Adjacent note (same resume surface, distinct defect — not part of this fix): V2 agent metadata restore re-registers every persisted Open spawn edge across each resume, and the V2 LRU eviction path (core/src/agent/control/residency.rs:117-151, try_unload_one_resident) unloads the resident thread without releasing the registry slot, so the three count surfaces (registry total_count, list_agents = tree ∩ live threads, env-context edges) can diverge and inflate monotonically over repeated resumes. The mainstream V2 spawn path is uncapped so this does not block spawns; it shows up as wrong logical counts in list_agents/env-context. Mentioning for triage only — the reconciliation fix (derive model-visible counts from a single registry ∩ live-threads projection) belongs on its own issue. No unsolicited PR from me.