Skip to content

Bound v1 close_agent shutdown waits for unresponsive child agents #29937

Open
Open
Bound v1 close_agent shutdown waits for unresponsive child agents#29937
Labels
CLIIssues related to the Codex CLIbugSomething isn't workingsubagentIssues involving subagents or multi-agent featurestool-callsIssues related to tool calling

Description

@alygg77
alygg77
opened on Jun 25, 2026

Problem

multi_agent_v1.close_agent can block the parent turn indefinitely when a child agent accepts Op::Shutdown but its session loop never reaches termination.

This makes cleanup unsafe to put on the critical path: a parent that is otherwise done can become stuck for hours just trying to close an unresponsive child.

Related existing issues:

Fresh reproduction evidence

Observed in a local native multi-agent Codex session on 2026-06-24.

Environment:

  • parent session: 019ef523-5927-74e2-b687-369f17ae6bbc
  • child session: 019ef7e2-c130-72d1-9b3e-280ff9c36e90
  • child role/nickname: native explorer, Hilbert
  • platform: macOS arm64
  • parent CLI around run: 0.138.0, child CLI around run: 0.141.0, current local CLI after update: 0.142.0

Timeline from local rollout JSONL:

2026-06-24T04:28:15Z spawn_agent -> Hilbert 019ef7e2-c130-72d1-9b3e-280ff9c36e90
2026-06-24T04:48:42Z wait_agent timeout_ms=1000  -> {"status":{},"timed_out":true}
2026-06-24T04:50:29Z wait_agent timeout_ms=10000 -> {"status":{},"timed_out":true}
2026-06-24T04:50:46Z wait_agent timeout_ms=60000 -> {"status":{},"timed_out":true}
2026-06-24T04:51:56Z close_agent Hilbert, plus cleanup closes for two sibling agents
2026-06-24T11:56:27Z close_agent returned only after user abort

The tool outputs reported approximately:

aborted by user after 24930.6s
aborted by user after 24930.5s

So the close path occupied about 7h04m wall-clock and only ended because the user aborted it.

The child transcript contained session metadata, task start/abort lifecycle, and no useful assistant/tool result.

Current code path

Current main still appears to have this shape:

state.send_op(agent_id, Op::Shutdown {}).await
thread.wait_until_terminated().await

The wait on thread.wait_until_terminated() is unbounded. If shutdown/task abort/lifecycle cleanup never completes, close_agent blocks the parent tool call indefinitely.

Desired behavior

close_agent should not wait forever for a child runtime to finish shutdown.

Suggested behavior:

  • submit shutdown as today
  • bound the wait for session-loop termination
  • if the timeout fires, return a structured timeout/error to the parent instead of blocking indefinitely
  • do not remove/release/forget the child if termination was not observed, so the still-live thread remains inspectable/retryable and its slot is not falsely freed
  • normal completed shutdown should keep existing cleanup behavior

Candidate patch

Because external code contributions are by invitation only, I am filing this as an issue/proposal rather than expecting an unsolicited PR to be accepted.

I pushed a minimal candidate implementation in a public fork for reference:

The candidate patch:

  • adds a bounded v1 close-agent termination wait
  • returns the existing request-timeout error on termination timeout
  • leaves the live thread tracked on timeout
  • adds a deterministic regression test with a child task whose abort hook blocks

Validation on candidate patch

Commands run locally:

cargo fmt --check
cargo test -p codex-core close_agent_times_out_when_child_session_does_not_terminate
cargo test -p codex-core close_agent_submits_shutdown_and_returns_previous_status
RUST_MIN_STACK=8388608 cargo test -p codex-core shutdown_agent_tree_closes
RUST_MIN_STACK=8388608 cargo test -p codex-core tool_handlers_cascade_close_and_resume_and_keep_explicitly_closed_subtrees_closed
RUST_MIN_STACK=8388608 cargo test -p codex-core spawn_agent_releases_slot_after_shutdown

Notes:

  • cargo fmt --check passed, with stable rustfmt warnings about the repo config key imports_granularity being unstable.
  • On this local macOS/Rust 1.95 setup, some existing agent-control tests stack-overflow without RUST_MIN_STACK; I verified upstream main shows the same stack overflow for spawn_agent_releases_slot_after_shutdown, so that appears pre-existing in this environment rather than introduced by the candidate patch.

Why this matters

Cleanup should not be able to consume hours of wall-clock time after useful work is done. Once wait_agent has already timed out repeatedly, calling close_agent should be a bounded cleanup operation, not a potentially indefinite parent-turn blocker.

Activity

  1. github-actions
    added
    bugSomething isn't working
    CLIIssues related to the Codex CLI
    subagentIssues involving subagents or multi-agent features
    tool-callsIssues related to tool calling
    on Jun 25, 2026

  2. guidedways commented on Jun 28, 2026

    @guidedways
    guidedways
    on Jun 28, 2026
    Contributor

    Seeing this every time, but the larger issue remains - agents get 'stuck' randomly and often: #30400

  3. shahidbeig-a11y commented on Jun 30, 2026

    @shahidbeig-a11y
    shahidbeig-a11y
    on Jun 30, 2026

    Same issue

    What issue are you seeing?

    A Codex Desktop subagent can stall during startup/tool discovery, then later be marked closed in the spawn-edge table while its child thread remains visible/unarchived and incomplete. After that, close_agent hangs instead of treating the agent as already closed or cleaning up the orphaned child thread.

    This creates a confusing state for the user: another agent cannot close the subagent, the child appears live/stuck, but the evidence shows it never actually ran user work.

    Environment

    • Product: Codex Desktop
    • Platform: macOS
    • Bundled Codex version seen in thread metadata: 0.142.4
    • Model: gpt-5.5
    • Affected child thread / agent id: 019f195b-3609-76a0-bc62-a65e208feee0
    • Parent thread id: 019f1684-c843-7ad1-94f0-bf77bd5acbb2
    • Child nickname: Singer
    • Child role: explorer

    Steps to reproduce

    1. From a Codex Desktop thread, spawn a read-only explorer subagent.
    2. The child starts while MCP/tool discovery is still initializing.
    3. The child stalls before producing any assistant/tool output.
    4. Later, shutdown/close is attempted.
    5. The spawn edge becomes closed, but the child thread row remains unarchived and visible.
    6. A later close_agent call against the same child id hangs.

    Actual behavior

    The child thread rollout contains only four records:

    session_meta
task_started
response_item message: <turn_aborted>
event_msg turn_aborted

  4. LmyNBL commented on Jul 8, 2026

    @LmyNBL
    LmyNBL
    on Jul 8, 2026 – with ChatGPT Codex Connector

    Adding #31036 as additional corroborating Desktop/app-server evidence for this same failure mode.

    I rechecked current main, and the relevant v1 close path still appears unbounded:

    • multi_agent_v1.close_agent awaits session.services.agent_control.close_agent(agent_id).
    • AgentControl::shutdown_live_agent in codex-rs/core/src/agent/control/legacy.rs sends Op::Shutdown and then awaits thread.wait_until_terminated() without a timeout.

    So #31036 looks like another instance of the same root problem: once an interrupted/nonresponsive subagent never reaches termination, close_agent can block the parent turn indefinitely.

    I agree with the bounded shutdown-wait direction. The main semantic question seems to be timeout behavior: should close_agent return a timeout while leaving the live thread tracked for retry/inspection, or should it continue through cleanup/removal after the bounded wait? Either would be much better than an unbounded parent-turn block; keeping the thread tracked seems safer if termination was not actually observed.

  5. tachyon-beep commented on Jul 26, 2026

    @tachyon-beep
    tachyon-beep
    on Jul 26, 2026

    This might be the root cause of an FD exhaustion on linux bug. Subagents aren't being cleaned up and holding their FD's open.

    It’s the Codex app-server, PID 2877. Not the repo, and not global Linux exhaustion.
    Live evidence:
    Soft limit: 1,024 FDs
    Observed: 928–930 open
    619 pipes — every one maps to child-process stdin/stdout/stderr
    207 pidfd handles
    204 duplicate MCP-server processes:
    [snip MCP list]

    Dozens of old session JSONL files also remain open.

    Ed:

    OK, Codex kept investigating on its own (so grain of salt etc):
    Because Codex’s cleanup is tied to thread shutdown, not turn/subagent completion—and that shutdown transition is not happening.

    The chain is:

    task_complete / turn_aborted → thread remains loaded and subscribed → per-thread McpConnectionManager remains alive → Codex retains the three stdio pipe ends and PID handle → MCP process never receives EOF → nothing exits.

    Live evidence from this app-server:

    • 34 loaded threads
    • 30 have already completed or aborted
    • 27 are subagents
    • Only one thread/unsubscribe request was received
    • All 34 remain unarchived

    Codex 0.145.0 automatically subscribes clients when starting/resuming threads. Its documented 30-minute unload timer only begins after the last subscriber unsubscribes. Turn completion does not count. [Exact 0.145.0 lifecycle documentation](https://github.com/openai/codex/blob/rust-v0.145.0/codex-rs/app-server/README.md)

    The MCP shutdown implementation itself was fixed to terminate children when session shutdown actually runs. [Merged shutdown fix](#19753) But completed subagents are not reliably reaching that shutdown path—an acknowledged lifecycle bug that remains open upstream. [Subagent MCP leak](#17574)

    So the short answer is: Codex marks the work finished but keeps the owning session object alive. The MCP processes and pipes therefore still have a legitimate live owner and cannot clean themselves up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIbugSomething isn't workingsubagentIssues involving subagents or multi-agent featurestool-callsIssues related to tool calling

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions