Bug: Thread history data loss on restart — async persistence not flushed on shutdown (RolloutRecorder, SQLite WAL, session_index) #16644

Summary

Thread history is silently lost after app restart (reported in #16599) because three async persistence layers can all have in-flight writes when the process terminates. There is no shutdown barrier that waits for them to complete.

Root Cause

The persistence pipeline for a session looks like this:

App → RolloutRecorder (async mpsc, 256-item buffer) → background writer → JSONL
    → StateRuntime (SQLite in WAL mode) → state.db
    → session_index (append-only file)

All three writes proceed concurrently and none are guaranteed to flush before process exit.

1. RolloutRecorder drops buffered items on shutdown

File: codex-rs/rollout/src/recorder.rs

Items are queued into a bounded 256-item tokio::sync::mpsc channel and consumed by a background writer task. If the process is killed (SIGTERM, system restart, OS interruption during a permission prompt) before that task drains the channel, the queued items are dropped. There is no sync_all() / fsync() and no mechanism to await the task's completion before exit.

2. SQLite WAL not checkpointed on process exit

File: codex-rs/state/src/ (STATE_DB_VERSION = 5)

The state database runs in WAL mode. Writes accumulate in the -wal file and are only checkpointed into the main database on a clean close or explicit PRAGMA wal_checkpoint. An abrupt exit leaves the WAL uncheckpointed, so on next startup thread metadata (timestamps, titles, status) reverts to the last successful checkpoint — which can lag behind the actual rollout by many transactions.

3. Incomplete session_index write breaks sub-agent thread discovery

File: codex-rs/rollout/src/session_index.rs

The session index is append-only and backward-scanned at startup by find_thread_path_by_id_str(). Sub-agent thread IDs are only discoverable if their index entry was fully written and flushed. A mid-write process exit produces a truncated or absent entry, making sub-agent threads completely invisible on restart — even though their JSONL rollout files still exist on disk. This explains why sub-agent histories disappear entirely while the main thread partially recovers.

Proposed Fix

On graceful shutdown, in order:

1. Drop the sender half of the RolloutRecorder channel and await the writer task to fully drain before returning
2. Call PRAGMA wal_checkpoint(TRUNCATE) on the SQLite connection before closing the pool
3. Ensure session_index appends call sync_all() after each write, or are sequenced behind the recorder flush

For non-graceful exits, a startup recovery pass that reconciles a partial JSONL rollout via reconcile_rollout() against the WAL could recover data that's on disk but not yet indexed.

References

• User report: Lost part of thread history and all subagent thread history after Codex App restart. #16599