Skip to content

Subagent spawns duplicate the full parent session history into each child rollout file (quadratic disk growth) #36557

Open
Open
Subagent spawns duplicate the full parent session history into each child rollout file (quadratic disk growth)#36557
Labels
CLIIssues related to the Codex CLIbugSomething isn't workingperformancesessionIssues involving session (thread) management, resuming, forking, naming, archivingsubagentIssues involving subagents or multi-agent features

Description

@Iceorbz
Iceorbz
opened on Aug 2, 2026

Summary

When a session spawns subagents, each child's rollout .jsonl under ~/.codex/sessions/ begins with a complete copy of the parent session's history. Compaction events additionally re-write the entire replacement_history into the same log. For a long-lived session that spawns many subagents, disk usage grows roughly quadratically with session length.

Impact (real numbers from one machine)

  • One long-running thread (~3 weeks old, ~100 subagent spawns, ~1,400 compaction events) produced 37 rollout files over 800MB each in a single day — ~30GB/day, measured at ~97% duplicated bytes.
  • ~/.codex/sessions/ reached 42GB, and the 1TB disk hit 3.8GB free before the pattern was found.
  • Session profiling: 93 of 101 rollout files created that day traced to the same parent thread.

Repro

  1. Start a session; build up a few hundred KB of history.
  2. Spawn several subagents (any task).
  3. Observe each new rollout-*.jsonl starts with the parent's full serialized history; sizes grow with parent history length, not child work.
  4. Trigger compaction repeatedly; observe replacement_history re-written in full each time.

Suggested fixes

  • Reference the parent rollout by ID + offset instead of copying history into children.
  • Store compaction replacement_history as a delta or in a separate file, not appended in full to the live log.
  • Alternatively: transparent zstd compression of rollout files (we measured 10–20× on this content).

Environment

  • Codex CLI with Codex Desktop, macOS (Darwin 25.5), model gpt-5.6-terra, subagents used heavily via the native multi-agent feature.

Activity

  1. github-actions
    added
    bugSomething isn't working
    CLIIssues related to the Codex CLI
    subagentIssues involving subagents or multi-agent features
    sessionIssues involving session (thread) management, resuming, forking, naming, archiving
    performance
    on Aug 2, 2026

  2. github-actions commented on Aug 2, 2026

    @github-actions
    github-actionsbot
    on Aug 2, 2026 – with GitHub Actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  3. Hevesilaci commented on Aug 20, 2026

    @Hevesilaci
    Hevesilaci
    on Aug 20, 2026

    I can reproduce this on Codex Desktop on Windows with a long-lived development thread using subagents.

    Measured local session storage:

    • Platform: Windows
    • %USERPROFILE%\.codex\sessions: approximately 68.0 GB logical size
    • Session JSONL files: 231
    • thread_source = subagent: 225 files / 62.18 GB
    • thread_source = vscode: 4 files / 0.63 GB
    • Direct children of one long-running parent thread: 179 files / 62.17 GB
    • Parent thread ID: redacted
    • Many individual child rollout files are approximately 500–540 MB
    • The largest observed rollout was approximately 648 MB
    • No subagents were running when the measurements were taken.

    The growth is strongly concentrated on days with heavy multi-agent use. For example:

    • 2026-07-29: 37 files / 11.24 GB
    • 2026-08-08: 31 files / 10.91 GB
    • 2026-08-10: 34 files / 10.50 GB
    • 2026-08-11: 41 files / 14.48 GB
    • 2026-08-13: 19 files / 6.21 GB

    Inspecting the rollout session_meta showed that the large files are subagent sessions and that 179 of them point to the same parent thread.

    NTFS compression did not materially help:

    • Logical data: 68,006,759,077 bytes
    • Compressed storage: 62,553,317,376 bytes
    • Ratio: approximately 1.1:1

    This has a significant practical impact because normal multi-agent development can consume 10+ GB of local disk in a single working day, even though the actual repository itself is comparatively small.

    It would be very useful if completed subagent rollouts could either:

    • reference parent history instead of serializing it again,
    • be automatically garbage-collected after completion,
    • be stored using effective compression/deduplication,
    • or have a supported cleanup/retention setting in Codex Desktop.

    A disk-usage warning in the Codex UI would also help, because this growth is otherwise silent until the system drive becomes low on space.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIbugSomething isn't workingperformancesessionIssues involving session (thread) management, resuming, forking, naming, archivingsubagentIssues involving subagents or multi-agent features

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions