Codex CLI resume can SIGKILL/OOM on huge rollout dominated by repeated compacted records #30932

Summary

A long-running Codex CLI thread became effectively unrecoverable because the local rollout JSONL grew to ~19.1 GiB, dominated by repeated compacted records. Running codex resume <thread-id> on the affected thread caused unbounded memory growth and previously ended with macOS reporting Killed: 9 from the wrapper command that invoked the real Codex binary.

This report is intentionally sanitized. I am not attaching the raw rollout because it contains private transcript contents, local project details, credentials, and tool output. The data below is limited to file sizes, record counts, command shape, process behavior, and the successful recovery method.

Environment

• Codex CLI: codex-cli 0.142.5
• Platform: macOS arm64
• Physical memory: 64 GiB
• Invocation shape: codex resume <thread-id> --no-alt-screen
• The local shell wrapper also passed --enable hooks and a notify callback, but codex --version and codex --help worked normally, so the failure correlated with resume/history hydration rather than wrapper startup.

Observed impact

• The original user-visible failure was a terminal Killed: 9 while trying to resume a critical long-running session.
• A controlled resume probe against the same thread did not reach a usable prompt after ~86 seconds.
• During that probe, the native Codex process RSS rose from about 2.7 GiB at ~39 seconds to about 10.1 GiB at ~86 seconds, with no loaded UI yet.
• The probe was terminated before macOS killed it again.
• Smaller/control sessions loaded normally.

Local evidence from the affected rollout

Affected rollout file, sanitized path:

~/.codex/sessions/<yyyy>/<mm>/<dd>/rollout-<timestamp>-<thread-id>.jsonl
size: 20,532,797,699 bytes
lines: 94,096

Top-level record-size breakdown from a streaming scan:

lines=94096
bytes=20532797699
big_lines_gt_1mb=320
big_bytes=20318698662

compacted     count=304   bytes=20251559724  max_line=76242052  big=295
response_item count=60800 bytes=249376528    max_line=9628408   big=25
event_msg     count=32334 bytes=30898242     max_line=432263    big=0
turn_context  count=657   bytes=941236       max_line=1642      big=0
session_meta  count=1     bytes=21969        max_line=21969     big=0

The latest compacted record was enough to recover the thread:

last_compacted_line=93992
last_compacted_len=76234052
lines_after_last_compacted=104
total_lines=94096

Recovery workaround that succeeded

After making a full backup outside ~/.codex/sessions, a rescue rollout was generated from only:

1. the first session_meta record;
2. the latest compacted record;
3. all records after that latest compacted record.

Resulting repaired rollout:

size: 76,408,041 bytes
lines: 106

A duplicate test session using that same reduced shape loaded successfully in Codex CLI. Replacing the affected session file with the reduced shape also loaded successfully and displayed the active goal state. After the verification resume/quit, the repaired file remained small:

size: 76,451,950 bytes
lines: 119

The full original 19.1 GiB rollout was preserved as a backup, but the normal resume path became usable only after obsolete pre-latest-compaction records were removed from the active session file.

Reproduction shape

1. Run a long-lived Codex CLI thread/goal for days with many turns, tool calls, and repeated context compactions.
2. Let rollout JSONL accumulate many compacted records, each containing large replacement history.
3. Attempt codex resume <thread-id>.
4. Observe very high memory growth while Codex hydrates/replays the rollout; in this case the process previously ended as Killed: 9.
5. Build a reduced rollout from session_meta + latest compacted record + suffix; the reduced rollout loads.

Expected behavior

• Resume should not need to hydrate every obsolete compaction snapshot when a later compaction supersedes prior history.
• A single oversized local rollout should not be able to SIGKILL/OOM the CLI process.
• Codex should provide a built-in safe recovery path, such as “resume from latest valid compaction checkpoint + suffix,” without manual JSONL surgery.
• If full recovery is impossible, Codex should fail the specific thread with a clear error and preserve/export the raw rollout rather than crashing.

Actual behavior

• Resume attempted to load/replay enough of the 19.1 GiB rollout that the process consumed multi-GB memory before any usable UI appeared.
• The original resume attempt was killed by the OS.
• Manual repair was required: full backup, streaming scan to find the latest compacted record, and replacement of the active rollout with a checkpoint-plus-suffix slice.

Suggested fixes

1. Add a bounded resume path that materializes only the first session_meta, the latest valid compacted checkpoint, and the suffix after that checkpoint.
2. Treat older pre-latest-compaction records as archive data, not hot resume data.
3. Rotate/prune/archive rollout records after successful compaction while preserving an exportable full audit trail.
4. Add rollout byte/record warnings for long-running sessions before they reach multi-GB sizes.
5. Add a built-in codex debug/recover-session <thread-id> or similar command that produces a safe checkpointed continuation file.
6. Avoid reading huge JSONL histories into memory or constructing an unbounded in-memory vector for normal resume/list/read paths.

Related issues

This looks closely related to prior large-rollout reports, but this case adds a current CLI/macOS reproduction with a 19.1 GiB rollout where repeated compacted records account for ~20.25 GB of the file and latest-compaction-plus-suffix recovery was verified.

• Long threads are unrecoverable when rollout JSONL grow beyond resume/list limits #25215
• Codex Desktop meta-bug: unbounded session/turn state causes freezes, context bloat, and lost active-turn control #25779
• Codex app-server can grow to 30-40 GB when local rollout history is huge #29510

Privacy note

I cannot attach the raw rollout or line samples because they contain private user/project data. The counts, sizes, and recovery procedure above are sanitized.