Skip to content

Subagent fork sessions persist large JSONL histories indefinitely, causing severe ~/.codex disk bloat #30779

Open

Description

@msokolov-meta

What version of Codex CLI is running?

codex-cli 0.141.0

What subscription do you have?

API

Which model were you using?

gpt-5.5

What platform is your computer?

No response

What terminal emulator and version are you using (if applicable)?

No response

Codex doctor report

What issue are you seeing?

Codex appears to persist every subagent/fork as a normal local thread rollout under ~/.codex/sessions, including full parent history when the agent is created with full-history fork mode. These JSONL files do not appear to be automatically cleaned up when subagents complete.

On one real install, after multi-agent work:

  • ~/.codex/sessions: 54G
  • ~/.codex/sessions: 4,468 .jsonl files
  • ~/.codex/archived_sessions: 2.0G
  • ~/.codex/logs_2.sqlite: 16G
  • several individual rollout JSONL files are 400-700 MB
  • two parent threads have 548 and 1,797 spawned descendants respectively

This makes local history expensive to scan/load and causes large disk growth over time.

What steps can reproduce the bug?

What steps can reproduce the bug?

  1. Start a Codex thread and build up a sizable history.
  2. Spawn many subagents using full-history fork behavior, for example legacy fork_context: true or v2 fork_turns: "all" / default full-history behavior.
  3. Let the subagents complete.
  4. Inspect ~/.codex/sessions.

Useful inspection commands:

du -sh ~/.codex ~/.codex/sessions ~/.codex/archived_sessions ~/.codex/logs_2.sqlite ~/.codex/state_5.sqlite
find ~/.codex/sessions -type f -name '*.jsonl' | wc -l
find ~/.codex/sessions -type f -printf '%s %p\n' | sort -nr | head -20
sqlite3 -readonly ~/.codex/state_5.sqlite \
  "SELECT parent_thread_id, COUNT(*) FROM thread_spawn_edges GROUP BY parent_thread_id ORDER BY COUNT(*) DESC LIMIT 20;"

What is the expected behavior?

Closed subagents become archived/cleaned up.

Additional information

No response

Activity

  1. added
    CLIIssues related to the Codex CLI
    subagentIssues involving subagents or multi-agent features
    sessionIssues involving session (thread) management, resuming, forking, naming, archiving
    on Jul 1, 2026
  2. shrishmanglik commented on Jul 22, 2026

    @shrishmanglik

    Adding fork-side measurements that quantify this (Codex app 26.715.8383.0 / cli 0.145.0-alpha.27, Windows 11):

    • One day of sessions = 15.4GB across 52 JSONL files; 51/52 were legacy-history subagent rollouts (99.98% of bytes). 25 children forked from a single image-heavy parent accounted for 77.3% of the day.
    • Each fork re-materializes the parent's image-bearing history: 1,219 inline-image occurrences vs 241 unique payloads — a 20.95× duplication factor. Short children (7–13 min of work) landed at ~0.9GB each.
    • Config-schema check on 0.145.0-alpha.27 found no supported key for image persistence, rollout retention, or max-bytes, so this isn't user-mitigable via config.
    • Workaround that held for us: developer_instructions mandating fork_turns="none" (children receive a bounded task packet instead of forked history) + an external retention prune.

    Full attribution packet: #34061 (comment)

  3. jasonarend commented on Aug 6, 2026

    @jasonarend

    This is affecting Codex Desktop as an archive-retention problem too, not only active subagent rollouts.

    A macOS Desktop profile now has 1,641 archived threads using 147.97 GiB under ~/.codex/archived_sessions, with only 30 GiB left on the internal SSD. The history is durable project context, so deleting it is not a reasonable fix. There is currently no supported external/cold archive migration path; manual relocation risks stale rollout paths, unsupported symlinks, and cross-volume archive failures.

    I filed the focused external-storage request with measured impact and concrete acceptance criteria here: #37216

    A first-class retention and external archive tier would address both this archive bloat and the broader session-growth problem.

  4. Fire-Dragons commented on Aug 9, 2026

    @Fire-Dragons

    Adding a substantially larger macOS Desktop datapoint from a live storage/state audit.

    Environment

    • Codex Desktop: 26.730.61639 (build 6234)
    • Bundled CLI: 0.147.0-alpha.1.2
    • macOS 26.5.2 (25F84), arm64
    • Long-running parent threads with repeated native subagent workflows

    Filesystem attribution

    • Codex home: 447 GiB
    • sessions: 405 GiB
    • archived_sessions: 36 GiB
    • 718 JSONL files under the active sessions tree
    • 672 files identified as subagent rollouts: 389.90 GiB
    • 46 primary-thread rollouts: 15.11 GiB
    • 302 rollout files are larger than 1 GiB

    Correlating child rollout IDs with the current state_5.sqlite.thread_spawn_edges table:

    • mapped closed child rollouts still under active sessions: 288 files / 325.07 GiB
    • mapped open child rollouts: 236 files / 59.34 GiB
    • child rollouts no longer represented by a current spawn edge: 148 files / 5.48 GiB

    The current state DB contains 644 spawn edges: 382 closed, 262 open. This means the largest storage block is not live child work; it is completed child history that remains fully materialized under active sessions.

    Fork-time amplification sample

    One newly spawned child rollout reached 1,419,745,225 bytes and 161,632 JSONL records. Of those records, 161,472 carried timestamps from the exact spawn minute. The bulk consisted of inherited/replayed event_msg, response_item, turn_context, world_state, compacted, and session_meta records, while subsequent growth was slow.

    Three long-lived parents accounted for approximately 393.5 GiB of the session data. This supports the quadratic/full-history materialization behavior described in #36557, rather than normal child output growth.

    Expected behavior

    • A child fork should reference or delta the parent history instead of serializing the full accumulated history into every child rollout.
    • Terminal children should have a supported retention/GC path that does not require deleting live session files manually.
    • task_complete / close should leave durable lifecycle state that makes completed children safe to archive or prune.
    • Codex should expose a retention cap or warning before local rollout history can consume hundreds of GiB and leave the system disk at 99% usage.

    No transcript content, project identifiers, thread IDs, or machine-specific absolute paths are included here. The lifecycle-state mismatch from the same audit is being added separately to #25179.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIbugSomething isn't workingperformancesessionIssues involving session (thread) management, resuming, forking, naming, archivingsubagentIssues involving subagents or multi-agent features

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions