Skip to content

Codex Desktop app-server leaks child processes and writes excessive logs after crash/restart #26869

Open

Description

@andersaamodt

Summary

Codex Desktop on macOS appears to lose track of tool child processes across threads and then writes an excessive amount of local log/session state. The visible result was a Codex crash followed by high disk-write pressure, many stale child processes under codex app-server, syspolicyd launch-validation bursts, and eventually a system state where UI remained visible but launching executables/actions became unreliable until reboot.

I am not attaching raw ~/.codex/logs_2.sqlite or session JSONL files because they contain user prompts, local paths, cwd values, command strings, and private project metadata. The details below are sanitized from local diagnostics.

Environment

  • App: Codex Desktop
  • App bundle version: 26.602.40724 (CFBundleVersion 3593)
  • Codex Framework: 149.0.7827.54
  • Binary under load: /Applications/Codex.app/Contents/Resources/codex app-server --analytics-default-enabled
  • macOS: 26.3 (25D125)
  • Hardware: Apple Silicon, Mac16,11
  • SIP enabled

What happened

  1. Codex Desktop crashed.

    • Crash report: Codex-2026-06-07-014424.ips
    • Exception: EXC_BREAKPOINT / SIGTRAP
    • Faulting thread: main CrBrowserMain thread
    • Follow-on renderer abort: Codex (Renderer)-2026-06-07-014456.ips
  2. After restart/reboot, Codex app-server generated repeated macOS resource diagnostics for disk writes:

    • codex_2026-06-07-021428_[host-redacted].diag
      • Event: disk writes
      • Writes: about 2147 MB over 74s
      • Heaviest stack ended in pwrite
    • codex_2026-06-07-022252_[host-redacted].diag
      • Event: disk writes
      • Writes: about 8589 MB over 505s
      • Heaviest stack ended in pwrite
  3. Active codex app-server had many long-lived child processes from multiple completed/interrupted tool calls.

    • Examples of child process types observed under the app-server PID:
      • rg processes
      • /bin/zsh -lc ... shells
      • Python worker processes
      • multiple node_repl processes
    • Some were tiny/sleeping but persisted for minutes to tens of minutes.
  4. Codex's process registry did not match the actual OS process state.

    • ~/.codex/process_manager/chat_processes.json contained about 187 records.
    • About 107 records had osPid: null.
    • None of the non-null registered osPid values matched currently live OS PIDs at the time of inspection.
    • The real codex app-server still had live children not represented accurately by that registry.
  5. Local Codex log state was very large and actively being written.

    • ~/.codex/logs_2.sqlite: about 1.6 GB
    • ~/.codex/logs_2.sqlite-wal: about 39 MB
    • ~/.codex total: about 3.9 GB
    • ~/.codex/archived_sessions: about 1.7 GB
    • ~/.codex/sessions: about 264 MB
    • SQLite log DB contained about 535k rows, including about 252k TRACE rows and about 208k INFO rows.
    • Top log targets included websocket/SSE/otel logging paths, e.g. codex_api::endpoint::responses_websocket, codex_otel.log_only, and codex_otel.trace_safe.
  6. macOS syspolicyd logged launch-validation bursts during the incident.

    • Repeated messages included dispatch_mig_server returned 268435459 and validation-category warnings for rapidly changing PIDs.
    • This correlated with many command-line children being spawned by Codex.
    • User-visible symptom: the desktop looked normal, but executable launches/actions stopped working until reboot.

Expected behavior

  • Completed/interrupted tool calls should be reaped reliably.
  • Process manager state should match live OS children or cleanly mark completed processes.
  • Codex should not keep stale rg, shell, Python, or node_repl children alive after turns are complete.
  • Local log/session writes should be bounded enough not to trigger macOS disk-write resource diagnostics during normal Desktop use.
  • A Codex crash/restart should not leave the app-server in a state that causes system-wide launch/security pressure.

Actual behavior

  • Codex app-server retained many child processes from multiple threads.
  • Process registry became stale/inaccurate.
  • Codex wrote gigabytes of file-backed state within minutes, triggering macOS diagnostics.
  • macOS launch validation (syspolicyd) was hammered by repeated process launches.
  • The system eventually required reboot because actions/executables stopped working reliably.

Privacy note

I intentionally did not include raw .sqlite, .jsonl, crash report bodies, cwd values, local usernames, hostnames, or command strings that reveal private project names. I can provide raw diagnostics through a private support channel if needed.

Activity

  1. added
    bugSomething isn't working
    appIssues related to the Codex desktop app
    app-serverIssues involving app server protocol or interfaces
    on Jun 7, 2026
  2. andersaamodt commented on Jun 7, 2026

    @andersaamodt
    Author

    Additional recurrence: this happened a third time on 2026-06-07 after another restart. Post-restart checks again did not point to memory pressure: no swap/compressor pressure and the internal SSD reported SMART Verified with about 60 GiB free. The recurring signal remains launch/process pressure around Codex Desktop: very large syspolicyd validation bursts during/after restart, Codex app-server validation warnings, prior Codex CLI diagnostic reports showing multi-GB disk writes in minutes, and stale process-manager records not matching live OS PIDs. I installed a local low-overhead guard to capture future recurrence metrics without attaching raw logs. Raw local logs still appear privacy-sensitive and are not being shared publicly.

  3. andersaamodt commented on Jun 7, 2026

    @andersaamodt
    Author

    Additional evidence from another recurrence on 2026-06-07 around 03:14 PDT: a local guard captured 7 Codex app-server processes and 14 direct children shortly before the restart, plus a fresh Codex diagnostic. /Library/Logs/DiagnosticReports/codex_2026-06-07-031212_*.diag reports Codex PID 18723 dirtied 2147.48 MB over 254 seconds, with the heaviest stack in pwrite. A 10-second unified-log slice from 03:10:00-03:10:10 contained 3166 syspolicyd dispatch_mig_server returned entries and 3168 validation-category-policy entries for rapidly changing PIDs. Shutdown stall collection at 03:14 captured Codex browser_crashpad_handler processes, consistent with Codex crashing/restarting during the degraded state. Firefox appeared to hang/crash from the user's perspective, but no fresh Firefox diagnostic report was produced after 03:10, so it looks like collateral launch/system degradation rather than an independent Firefox crash. Raw logs are still not being attached because they contain local/private context.

  4. andersaamodt commented on Jun 8, 2026

    @andersaamodt
    Author

    Additional evidence from a macOS launch/input freeze investigation (sanitized, no prompt/log payloads):

    • Environment: macOS 26.3. Symptom: after reboot, third-party/non-system app launches could hang indefinitely while existing system apps still partly worked. Firefox bounced in Dock; Terminal/shell-launched non-system runtimes also stalled; Codex UI/app-server stayed alive enough to continue some work.
    • The system-level failure mode was non-system executable launch assessment stalling before program startup. A sampled Codex child process was a bundled runtime python3 under ~/.cache/codex-runtimes/... with a tiny RSS footprint and its stack entirely at _dyld_start.
    • Reproduced outside Codex parent context: a one-shot launchctl submit job running a copied Codex node binary stayed in launch/running state instead of exiting immediately. Apple-signed system binaries like /usr/bin/true and /bin/mkdir still launched quickly. This points to macOS syspolicy/dyld/provenance assessment under load, not a Python/Node script bug.
    • During the bad window, syspolicyd was hot for long periods and the system accumulated many tiny pre-main or zombie children. Codex app-server was a major amplifier: it had dozens of direct child processes (rg, node_repl, bundled runtime Python/Node, Computer Use helper) stuck with very small RSS. Firefox also accumulated plugin-container zombies.
    • Codex local process-manager state contained about 196 remembered tool records, many from old chats/repos, mostly rg. Only a few remembered PIDs were live, so this did not look like a simple relaunch queue, but it is a strong signal that Codex should prune/reconcile stale tool-process state and avoid child-launch storms after reboot or after launch failures.
    • The local root cause had multiple non-Codex producers too: stale launch agents were crash-looping/misconfigured, including Node helpers pointed at missing files and stale app-start launchers. After removing/repairing those launch agents and letting the backlog drain, syspolicyd returned near idle and Codex/Homebrew Node launch latency returned to ~0.05s in quiet samples.

    Codex-specific recommendation: add backpressure/timeouts around app-server child launches on macOS when children remain tiny and stuck at _dyld_start; aggressively reap/kill and mark failed rather than continuing to spawn; prune stale ~/.codex/process_manager/chat_processes.json records; and avoid repeated Computer Use/helper launches while syspolicy validation is saturated. Codex was not the only root cause in this incident, but it amplified the system-wide launch wedge enough that the whole GUI/input session became unrecoverable without a reboot.

  5. andersaamodt commented on Jun 8, 2026

    @andersaamodt
    Author

    Follow-up evidence from a later recurrence on macOS (2026-06-07 around 22:24-22:25 PT), sanitized to avoid raw prompts/payloads:

    • syspolicyd repeatedly sat around ~51-53% CPU while Codex-launched child processes remained at extremely low RSS (typically 32 KB), consistent with being stuck pre-main / before interpreter startup.
    • Live examples included Codex child jobs such as Homebrew Node/npm, node ... prettier --check ..., rg ..., and short shell helpers. The parent of the live stuck children was /Applications/Codex.app/Contents/Resources/codex app-server --analytics-default-enabled.
    • Unified log during the same window showed a flood of com.apple.Gatekeeper.PolicyViolation telemetry from syspolicyd, com.apple.syspolicy.exec messages with dispatch_mig_server returned 268435459, and messages of the form PID ... violates validation category policy. This will likely result in a block in the future. macOS privacy redacted the executable path in the log, but simultaneous ps output tied the active low-RSS launch queue to Codex app-server descendants.
    • System binaries continued to launch normally while non-system developer tool/app-bundle launches queued or wedged. In bad cases this escalated to Firefox/app launches bouncing forever and user input/app startup appearing globally broken until reboot.

    This points to Codex Desktop needing backpressure/reuse/throttling around child process launches and/or explicit detection of macOS launch-assessment saturation (syspolicyd / syspolicy.exec) so it stops scheduling more non-system executable launches while the OS launch path is wedged. Raw command payloads are intentionally omitted because they can include user prompt text and local paths.

  6. andersaamodt commented on Jun 8, 2026

    @andersaamodt
    Author

    Cross-linking related reports that look like the same or adjacent failure domain. I’m not claiming they all have one root cause, but they point at the same macOS launch-assessment / signing / helper-process pressure area:

    The common engineering hypothesis from this report plus those threads: Codex Desktop should apply backpressure/reuse/throttling around spawned helpers and tool-server processes, and should detect when macOS launch assessment (syspolicyd / syspolicy.exec / trust validation) is saturated before scheduling more non-system executable launches. It would also help to verify notarization/signing/caching behavior for all shipped helper binaries, especially Computer Use / SkyComputerUseClient and CLI/runtime binaries that may be spawned repeatedly.

  7. andersaamodt commented on Jun 8, 2026

    @andersaamodt
    Author

    Follow-up with a more specific recovery experiment from the 2026-06-08 recurrence, again sanitized to avoid prompt text or local command payloads:

    • During the incident, Apple-signed system binaries (/usr/bin/true, /bin/ls) launched immediately, while a Codex-bundled helper launch (/Applications/Codex.app/Contents/Resources/node_repl --version) hung. That narrowed the failure to non-system executable launch assessment rather than global CPU, disk, or shell failure.
    • The Codex app-server process had a large descendant tree at the time, including many tiny low-RSS children stuck before useful startup. Examples included Codex-bundled helpers, Node helper launches, rg, Playwright/Chromium helpers, and site-status/build helpers launched through active Codex work.
    • I terminated only stale low-RSS pre-main Codex-descended children, leaving the Codex app/server itself running. Within about 30 seconds, syspolicyd CPU dropped from roughly 56% to under 1%, and the count of Codex low-RSS stuck children fell to zero. No reboot and no Codex quit was needed for that observed drain.
    • Active Codex automations were checked: the daily automation was not firing during this 01:xx local incident, and the hourly automation was paused. The trigger looked like normal concurrent interactive Codex workload fanout rather than a scheduled automation.

    This is stronger evidence that Codex Desktop needs a macOS launch-assessment backpressure mechanism, not only orphan cleanup after the fact: if spawned children remain tiny/pre-main or syspolicy validation is hot, Codex should stop scheduling more non-system executable launches, reuse existing helper processes where possible, and aggressively fail/reap the stuck child jobs instead of continuing to fan out.

  8. andersaamodt commented on Jun 11, 2026

    @andersaamodt
    Author

    Additional live incident evidence from 2026-06-10 ~20:00 PDT:

    • User-visible symptom: Firefox pages spun forever / app launches stalled while Codex itself could still run existing tool sessions.
    • syspolicyd was repeatedly at ~53-67% CPU while many newly launched non-system helpers sat at ~32 KB RSS, apparently pre-main / blocked in launch assessment.
    • Producers observed during the incident included Codex app-server child helpers (rg, node/node_repl, Computer Use completion helpers), Firefox plugin-container launches, and a local Headquarters deployment automation fanout (auto-template, publish-status/site-status, and auto-push/publish helpers).
    • Terminating stuck helper leaves alone reduced the 32 KB queue but did not recover the system while producers kept respawning. Restarting syspolicyd while fanout continued also immediately re-saturated it.
    • The reliable recovery sequence was: stop/kill the fanout producers, archive active background Codex threads, clear stuck helper leaves, then admin-restart syspolicyd. After that: syspolicyd returned to 0.0% CPU and the low-RSS stuck queue went to zero.
    • An admin sample of the hot syspolicyd process showed the active path under syspolicyd.executions.mig in Security.framework code-signing verification, especially SecCodeCopySigningInformation, Security::CodeSigning::SecStaticCode::signingInformation, validateDirectory, CMSDecoderCopySignerStatus, and timestamp/cert parsing (decodeTimeStampTokenWithPolicy). This supports the diagnosis that the stall is launch/code-signing assessment saturation, not DNS/network/SSD/memory pressure.

    Local mitigation added on my side: I throttled one local app's background publish/status fanout and added a macOS launch-pressure guard so it defers passive automation when syspolicyd CPU or 32 KB launch-stall counts are high. That helps reduce one producer, but Codex still appears able to create the same pressure via multiple active/background threads and completion/helper launches.

  9. jiansongy commented on Jun 30, 2026

    @jiansongy

    Independent reproduction on Intel macOS — same disk-write/log signature, escalating to a full WindowServer watchdog kill (whole-screen freeze)

    Corroborating this on different hardware and a much newer build. Same 2147 MB … pwrite disk-write signature and TRACE-dominated logs_2.sqlite, but here it escalated past "launches stall" to a WindowServer userspace_watchdog_timeout — the entire screen froze, twice in one day.

    Environment

    • Codex Desktop 26.623.70822 (CFBundleVersion 4559) — newer than the 3593 above, so the behavior persists in current builds
    • Codex Framework 149.0.7827.197; binary under load …/Codex.app/Contents/Resources/codex app-server --analytics-default-enabled
    • MacBookPro16,2, Intel x86_64, 8 logical CPUs, 16 GB — the reports here so far are Apple Silicon; this widens scope to Intel
    • macOS 26.5.1 (25F80), SIP enabled

    What macOS recorded

    1. codex_<ts>.diag — Event: disk writes, 2147.48 MB dirtied over 8825 s (~243 KB/s, ~10× the limit). Heaviest stack is SQLite WAL churn: sqlite3_exec → sqlite3_step → sqlite3_wal_checkpoint_v2 → pwrite (via sqlx_sqlite). ~/.codex/logs_2.sqlite was 301 MB / 80,685 rows, 71% TRACE (57k TRACE / 14k INFO / 8k DEBUG vs 1.2k WARN / 20 ERROR) over a 10-day span — overwhelmingly trace-logging write amplification, matching the codex_otel.trace_safe / *.log_only targets noted above.
    2. Two concurrent codex app-server instances at freeze time, one orphaned to launchd (PPID 1) since ~7 h earlier, never reaped, still holding logs_2.sqlite open and writing. ~38 Codex-owned processes / ~1.3 GB RSS incl. accumulated cua_node/node_repl + mcp-server children (matches Codex for macOS accumulates Computer Use / MCP helper processes and unreaped zombie children, causing HID lag and WindowServer/TCC stalls #25744).
    3. WindowServer userspace_watchdog_timeout → whole-screen freeze (the new/worse symptom). The spindump shows WindowServer was not CPU-spinning: its main thread used 0.2 s CPU over the 5.57 s watchdog window and was blocked in mach_msg the whole time (SkyLight → … → mach_msg_trap), CoreDisplay likewise blocked. Reason: WindowServer returned not alive … Display … not ready … Surface Use Counts: 2(1) 10(3) 11(1). It was starved, not buggy — it could not get a timely reply while the run queue was saturated. In the same stackshot the top CPU consumers were syspolicyd (3.0 s, in com.apple.security.syspolicy.yara / XProtect) and trustd (1.2 s) — consistent with Codex Desktop for macOS repeatedly triggers syspolicyd / trustd CPU and memory runaway #25719, driven by the constant new-process launches. Load average ≈ 470, ≈ 105 MB free RAM at that instant.

    So the separately-filed symptoms look like one chain on a memory-constrained Mac: app-server child/log leak (this issue) + CUA/MCP accumulation (#25744) saturate CPU/RAM/disk → syspolicyd/trustd spike (#25719) → WindowServer misses its watchdog deadline → it gets killed → full-screen freeze.

    Mitigations that worked here (until the next session): kill the orphaned app-server; prune old logs rows + VACUUM (301 → 110 MB); restart Codex (collapses 38 → 14 processes, 2 → 1 app-server). A user-facing toggle to disable --analytics-default-enabled or a size cap on logs_2.sqlite would remove the disk-write driver. Redacted .spin / .diag headers available on request.

  10. Veneyo commented on Aug 11, 2026

    @Veneyo

    Linux confirmation: stale node_repl helpers trigger an inotify-capacity warning

    I observed a closely related child-process lifecycle symptom on Linux. In this case, accumulated Codex helpers caused KDE to report exhausted inotify instance capacity.

    Environment

    • Fedora Linux 44
    • Kernel 7.1.5-201.fc44.x86_64
    • KDE Plasma 6.7.3
    • kde-inotify-survey 26.04.3
    • Codex/ChatGPT Desktop package 26.803.81509

    No usernames, hostnames, local paths, PIDs, project names, task contents, command arguments, or raw logs are included below.

    Observed state

    System limits:

    • fs.inotify.max_user_instances = 128
    • fs.inotify.max_user_watches = 268688

    KDE reported:

    Currently using 292% of instances and 2% of watches.

    A measurement shortly afterward found 399 per-process inotify descriptor references across the user session. At least 342 were associated with the ChatGPT/Codex process tree:

    • 36 node_repl processes: 216 references
    • 17 node processes: 102 references
    • ChatGPT/Codex parent and support processes: 24 references

    All 36 node_repl processes had the same Codex parent and had accumulated over roughly 12 minutes.

    The 399 count represents descriptor references, not necessarily 399 unique kernel inotify instances. The Codex parent held eight inotify descriptors; two sampled node_repl children each held the same subset of six descriptor numbers as the parent. That suggests descriptor inheritance, potentially from descriptors not marked close-on-exec. It may also amplify KDE's percentage calculation, but it does not explain away the accumulation of 36 helper processes.

    Restart result

    After fully quitting and relaunching the desktop app:

    • Total inotify descriptor references: 399 → 93
    • node_repl processes: 36 → 1
    • node processes: 17 → 1

    Additional normal tool use after the restart left only one node_repl/node pair, so I have not yet reproduced the accumulation deterministically. This suggests the app-server or process manager may first need to enter a degraded state.

    Impact and suggested investigation

    This produces a system-level warning and risks preventing unrelated applications from creating file monitors. Raising fs.inotify.max_user_instances would only mask the lifecycle issue.

    It may be useful to check:

    • whether completed or replaced node_repl/Node helpers are reliably reaped or reused;
    • whether parent inotify descriptors are missing FD_CLOEXEC/O_CLOEXEC;
    • whether crash, interruption, or runtime-replacement paths clean up the entire helper tree; and
    • whether stale helpers should be reconciled during desktop startup.

    This appears consistent with the stale child-process behavior described in this issue, with Linux inotify pressure as an additional user-visible consequence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    appIssues related to the Codex desktop appapp-serverIssues involving app server protocol or interfacesbugSomething isn't workingperformance

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions