migrate-rollouts leaves persisted subagent threads with empty projected history #38762
Description
What version of Codex CLI is running?
codex-cli 0.148.0-alpha.9. I also reproduced the relevant behavior on main at commit a7edf37cb46b5fc4d50bd03df8e5999a86f602eb.
What subscription do you have?
Not applicable. This happens during an offline local-store migration.
Which model were you using?
No model call is involved.
What platform is your computer?
macOS arm64.
What terminal emulator and version are you using (if applicable)?
Not relevant to the reproduction.
Codex doctor report
Not attached. I can reproduce this with a synthetic rollout fixture, and a doctor report would include unrelated local-state information.
What issue are you seeing?
After migrating a persisted legacy subagent, its raw rollout can still contain history while thread/read and resume surfaces show an empty thread.
For a migrated subagent, the migration currently writes subagent_history_start_ordinal equal to expected_ordinal, which is the end of the rewritten rollout. Projection then hides every record below that boundary. Since no existing record is at or beyond the end-of-file boundary, none of the migrated history is exposed as thread turns or items.
This is easy to notice with a named or otherwise persisted delegation thread that is still visible in thread/list: the thread exists and its rollout is non-empty, but opening it shows no previous turns.
What steps can reproduce the bug?
I used an isolated temporary Codex home and a synthetic legacy subagent:
All identifiers, names, timestamps, and rollout contents below are synthetic.
- Create a legacy rollout with
SessionSource::SubAgent, a compacted checkpoint, and one child-local user/assistant turn. - Make the thread persisted and visible to
thread/list. - Run
codex migrate-rollouts --apply --thread 00000000-0000-7000-8000-000000000002against the temporary home. - Inspect the migrated session metadata, then read or resume the thread through the app-server.
The migrated metadata has subagent_history_start_ordinal == number of rollout lines. The retained rollout is non-empty, but the projected thread has no active turns.
What is the expected behavior?
Migration should keep inherited or bootstrap context hidden where necessary, while preserving the subagent's own displayable turns for thread/read and thread/resume.
The boundary should point to the first displayable child-local record instead of always pointing to the end of the rewritten file. If ephemeral workers and persisted user-facing subagents intentionally need different behavior, the migration should distinguish those cases.
Additional information
I traced the behavior to these paths:
- The migration passes its final
expected_ordinaltorewrite_subagent_history_boundary. - That helper stores the value as
subagent_history_start_ordinal. - Materialization suppresses records below the boundary (source).
- The current migration test asserts both
boundary == lines.len()and an empty projected turn list (test).
That test makes the migration-layer behavior clear, but I am not sure it captures the expected app-server behavior for a persisted thread that a user can reopen. A focused regression test could migrate a named synthetic subagent, exercise thread/read or thread/resume, and verify that its child-local turn remains visible while inherited context stays hidden.
I found this while looking at the same migration path as #38761, but kept it in a separate report because this issue is about subagent history projection rather than legacy name preservation.
If this is not the intended behavior for persisted subagents, I'm happy to work on the focused fix and regression test if a maintainer invites a PR.
Activity
jdcodes1 commentedon Aug 18, 2026
Verified on
main@ 1f41cc5, and the live-creation path shows exactly what the migration should have written — the bug is a conflation of two different ordinals.The intended semantics. When a live subagent thread is created,
subagent_history_start_ordinalis set to the inherited-prefix length + 1 — the ordinal where the subagent's own visible history begins, just past the parent context it inherited:codex/codex-rs/thread-store/src/live_thread.rs
Lines 115 to 131 in 1f41cc5
What the migration writes instead. For a migrated subagent, the boundary is set to
expected_ordinal— the value returned by writing the rollout, i.e. the ordinal at the end of the entire rewritten file:codex/codex-rs/thread-store/src/local/rollout_migration.rs
Lines 553 to 561 in 1f41cc5
with
rewrite_subagent_history_boundarystamping that value verbatim into the headSessionMeta(rollout_migration/publish.rs#L157-L179). Since projection hides everything below the boundary and no record sits at or past end-of-file, the migrated thread projects as empty — your report, mechanically.Fix. The correct boundary is available at the same call site:
select_bounded_contextproduces the inherited-context prefix thatwrite_bounded_subagent_rolloutwrites first, so the boundary should be that prefix's item count + 1 (mirroring the live-creation formula), not the end-of-file ordinal. One value swap in theRolloutMigrationKind::Subagentbranch.Retroactive repair is possible. As you note, the raw rewritten rollout still contains the full history — only the head record's boundary is wrong. A repair pass (or a check in
migrate-rolloutsre-runs) can recompute the boundary with the corrected rule and rewrite the head line, restoring already-migrated subagent threads without data loss — same recoverability situation as the name regression in #38761, and worth shipping together since both aremigrate-rollouts --applyartifacts users can't fix themselves.Test shape: migrate a synthetic persisted subagent with N inherited-context items and M own-history records; assert the stamped boundary equals N+1 and
thread/readprojects exactly M records — plus the repair-pass variant against a store already migrated with the end-of-file boundary.feat(codex): expose internal Codex subagent sessions
guihao0728ster commentedon Sep 9, 2026
Persisted subagent migration: display restored, earlier durable history still missing
Related issue: #38762
Environment: macOS arm64; bundled codex-cli 0.153.4; desktop app 26.901.51231.
A persisted delegation thread created in July was subsequently used as a normal, long-running user-facing conversation. After legacy-to-paginated migration in September, it remained visible in the sidebar but its previous turns disappeared.
Observed metadata: history_mode=paginated, thread_source=subagent, subagent_history_start_ordinal=925. Records before ordinal 925 contained a completed child-local turn. The SQLite materializer omitted those records under the inherited-history condition.
After backing up both rollout and SQLite, an offline, length-preserving correction of the boundary to the first child-local turn, with invalidation of this thread's projection cursor, allowed the native reader to expose 1 user message and 27 public assistant messages. All pre-existing rollout body bytes were preserved. No other thread was modified.
However, the older months of turns are absent from the rewritten rollout. Its compacted replacement history retains many user prompts, but not the complete original assistant responses. A separate local task had preserved a read_thread response from before migration with three earlier turns, confirming that earlier visible turns existed before the rewrite.
Could maintainers clarify:
This report deliberately omits conversation text, thread identifiers, account data, private filesystem paths, credentials, and database attachments. Additional private diagnostics would require the owner's consent and a private support channel.
koelpinjudy commentedon Sep 9, 2026
Reproduced on ChatGPT Desktop for macOS with a real persisted, user-visible subagent thread (not a synthetic fixture). Bundled Codex version: codex-cli 0.153.4.
Observed:
Expected: persisted, user-visible subagent tasks should retain their child-local displayable turns after migration, while inherited/bootstrap context can remain hidden.
No session file or logs are attached because they may contain private conversation data.
AiurArtanis commentedon Sep 9, 2026
Additional Windows confirmation with an isolated-copy A/B test using bundled
codex-cli 0.153.4(Desktop client version reported in local logs:26.903.61454; observed September 9, 2026).Two persisted, user-visible forked tasks remain listed and idle, but opening them shows no historical conversation. Both have
thread_source: subagentandhistory_mode: paginated. Their existing rollout JSONL files are non-empty and every line parses successfully.Sanitized structural evidence (A/B are anonymous sample labels):
subagent_history_start_ordinalitem_completedline (1-based file line)next_rollout_ordinalthread_turns/thread_itemsFor both, the projection byte cursor equals the current rollout file size. Thus the cursor reports EOF although no turns or items are projected. Sample B also illustrates why an equality-only check (
boundary == current EOF) can miss this: the boundary is before the current EOF but still after all existing displayable items.Controlled local verification
thread/turns/listwithlimit: 100anditemsView: notLoaded: both returned 0 turns without an API error.userMessageand 35agentMessageitems for A, and 9userMessageand 36agentMessageitems for B (plus reasoning/file-change/compaction items).This confirms that the stored boundary suppresses recoverable existing display history under 0.153.4. It does not independently identify which migration/build originally wrote that value, and it does not establish that all older pre-compaction conversation text survives in the rewritten rollout.
Setting the boundary to 0 was a diagnostic control, not a proposed blanket repair: inherited/bootstrap context should remain hidden. A supported fix should compute the first child-local displayable boundary correctly and invalidate/rebuild the affected thread's already-advanced projection state, including for previously migrated persisted tasks. Rebuilding from the unchanged baseline metadata alone still produced an empty history in this test.
No conversation text, task names/IDs, private paths, account data, credentials, raw logs, or database/transcript attachments are included.
chenlixun1991-design commentedon Sep 9, 2026
We observed the same end-of-file history-boundary symptom on macOS with desktop version 26.903.61454 and bundled codex-cli 0.153.4. We would appreciate maintainer guidance on a data-preserving recovery procedure for already-migrated, persisted user-facing tasks.
Observations
A local audit on September 10, 2026 (Asia/Shanghai) covered 55 user-facing tasks:
subagent_history_start_ordinalequal to the record count, with the last ordinal one less than that boundary; projection cursors were at EOF.Isolated test of one affected task
We used backups and the native bundled app-server in isolated copies, with network access and production-file access blocked. No model turn was started, and no production database or rollout was modified.
The tested rollout contained 299 records (ordinals 0 through 298), with a history boundary of 299.
These are diagnostic candidates, not recommended recovery boundaries. Projected turns include synthetic/partial execution groups and are not equivalent to complete conversational exchanges.
For both candidates, paginated native API results were stable across repeated reads and an app-server restart. Exposed completed message text matched the corresponding source records by hash. No messages were fabricated or appended. The known-readable control task remained unchanged in the effective tests.
The unresolved issue is provenance: candidate A exposes two additional assistant messages before the first structurally complete user/assistant segment. Existing records do not establish whether those earlier fragments belong in the task's original child-local display history. Choosing candidate B could omit valid content; choosing A could expose inherited or partial content. Consequently, we stopped at isolated validation and did not apply either candidate to production.
Questions for maintainers
We have retained protected local backups and test evidence. We can provide additional sanitized structural details, but cannot publish raw histories or databases containing private work material.
ariccio commentedon Sep 9, 2026
Disclosure: This comment was generated by Codex, operating fully autonomously in the research and drafting of this comment. GitHub attributes it to
ariccio; Alexander selected the scope and approved this exact publication, but is not the speaker or author of Codex's technical claims.A bounded current-state audit found the reported EOF-boundary/empty-projection shape at substantial scale: 6,996 files had
subagent_history_start_ordinal == last retained ordinal + 1and zero stored projected items. This is not a count of proven lost conversations, and no boundary was edited. The original child-local histories must still be compared with pre-migration backup material.For persisted subagents, is there a supported way to distinguish intentionally hidden inherited context from child-local history that should remain visible? The source files being readable does not answer that application-history question.
Measurement, denominator, and version boundary
September 9, 2026; macOS 26.4 (25E246), arm64/APFS; Desktop 26.901.51231/build 8109, bundled CLI 0.153.4; separate CLI 0.153.2.
Across 8,553 individually stable inspected files, 7,045 readable metadata records contained a subagent history boundary. Of those, 6,996 boundaries equaled the next ordinal after the final retained record, and all 6,996 had an empty stored item projection. No observed boundary exceeded that next ordinal. The other 49 did not have an empty stored item projection in this observation.
This pass used bounded first/last-record inspection for most files, with source identity checks before and after each read. It did not reparse every middle record or establish whole-file hashes for the edge-only subset. Projection rows and selected rollout metadata came from separate WAL-aware read-only transactions; cross-database atomicity is not claimed. This task's changing rollout was excluded.
The source-level relationship described in the issue is consistent with this shape. This report does not independently reproduce every thread's UI-open/resume behavior, establish whether each thread was intended to be user-visible, or prove archival loss. A boundary before EOF is not automatically correct either; displayable-item placement must be checked separately.
Follow-up and why no manual fix is proposed
The planned historical comparison will account for child-local continuation records, copied ancestry, compaction/replacement history, oversized records, and material retained elsewhere. A record surviving in another file will not be called independently readable in its original child.
Moving a boundary cannot recreate omitted original records, and plausible-looking boundaries can expose different fragments. No repair, projection rewrite, rerun of migration, or retention change was performed or recommended.
Verbatim publication-request excerpt:
This comment adds the bounded structural measurement; the positive capacity report and developer thanks are kept in #42648 rather than repeated here.
mohyeeCT commentedon Sep 9, 2026
Windows desktop: boundary repair exposes surviving messages, but older conversation history remains absent
Additional Windows report. This corroborates the EOF-boundary mechanism in this issue and the earlier comment about missing pre-compaction replies.
Environment and impact
26.903.8094.0.codex-cli 0.153.4.One affected task: verified structural evidence
history_mode: paginated,source: vscode,thread_source: subagent.subagent_history_start_ordinal: 702.thread_turnsand zerothread_items.A September 2 local log recorded
rollout_item_count=37413for this task. The current 702-record representation is different, so this is not a count of deleted messages. It is a historical discrepancy requiring investigation.Controlled copy-only test
Each case used a separate disposable Codex home, the native engine, and a fresh derived history database.
The five groups in the boundary-5 case are not five complete exchanges: they include one partial segment with seven messages, an empty completion group, and three complete later turns. Reviewing the seven earlier messages shows continuity with the next retained user approval; the missing beginning and original pre-migration chronology have not been reconstructed.
Verification:
These are experimental diagnostic boundaries, not a blanket repair recommendation. The tests recover surviving records; they do not recreate older replies absent from the rewritten file.
Backup findings and recovery question
The two available Windows shadow copies already contain byte-identical copies of this shortened transcript and the same empty task projection. Their session/archive folders contain only the known transcript for the task; their rollout-migration folders are empty. Windows Backup reported no backup. No snapshot was restored. Protected post-incident backups remain available, but no complete pre-incident source has been found.
Could maintainers clarify:
thread_source: subagentbut has been used as an ordinary long-running desktop conversation?metze09 commentedon Sep 11, 2026
Confirmed on real migrated Windows threads + successful manual recovery.
Environment:
26.903.9818.0codex-cli 0.153.4I had three persisted subagent threads where the raw rollout JSONL still contained the conversation, but the Windows app showed either an empty thread or only the very newest part of the conversation. The observed behavior matches this issue very closely.
Observed signature
In all affected threads:
thread_source = subagentthread_history_projection_stateconsidered the rollout already processedthread_turns/thread_itemswere empty or started exactly aftersubagent_history_start_ordinalThree real examples (anonymized):
subagent_history_start_ordinalThread B was the clearest case:
Thread C showed the same pattern:
Thread A showed another useful confirmation:
So the first visible/materialized turn started exactly one ordinal after the incorrect boundary.
Important finding: conversation/model context was not actually lost
Before modifying anything, I tested the affected threads using
resume.Even when the Windows UI showed little or no previous history, Codex could still answer very specific questions about old project details that were not present in the visible conversation. I also verified that new information added to the resumed thread survived a full Codex restart and was still remembered afterwards.
So in these cases there were two separate layers:
An apparently empty migrated thread therefore does not necessarily mean the underlying conversation data is gone.
Manual recovery that worked
Important: this is a forensic/manual workaround, not a general fix. Back up the complete
CODEX_HOMEfirst, including SQLite WAL/SHM files and the original rollout JSONLs.1. Determine the correct boundary
Do not blindly set every affected thread to
1.The correct value is the first displayable child-local record / persisted child-history boundary. In my examples:
Thread B is particularly important because it demonstrates why blindly using
1is wrong: ordinals before6were inherited/bootstrap/compaction context and were not supposed to become normal child-visible turns.2. Correct only the head session metadata
In the first
session_metarecord of the affected rollout, change only:to the verified first displayable child-local ordinal. I did not alter the actual conversation/event records.
3. Reset only the derived projection for that thread
In
thread_history_1.sqlite:No rows belonging to other threads were touched.
4. Open the thread once in Codex
After reopening the thread in the Windows app, Codex rematerialized the rollout using the corrected boundary. The previously missing history became visible again.
Final verified results:
All three threads were then usable normally in the Windows app.
Important safety detail: WAL/runtime history
During testing I found an additional caveat that may matter for an automatic repair tool.
At one point
thread_history_1.sqlite(-wal)contained newer runtime turns with ordinals beyond the current rollout EOF. Those turns were resumable and visible to the model even though they had not yet been flushed into the rollout JSONL.Therefore a repair tool should not blindly delete/rebuild the projection while Codex is running or while newer runtime-only history exists.
Recommended order:
Otherwise a repair could restore old history while losing recent runtime-only turns.
Compaction
Recovery restored the normal displayable turns, but content older than an existing compaction checkpoint does not necessarily reappear as individual chat bubbles. Codex could still use that compacted context when resumed, but the UI can only reconstruct what remains representable as displayable history.
Suggested automatic repair
It seems feasible to detect this condition automatically:
subagentthread;subagent_history_start_ordinalat/near rollout EOF;A repair could then:
subagent_history_start_ordinal;This also independently confirms the analysis above that the migrated boundary should reflect the inherited-prefix length / first child-local history ordinal rather than the end-of-file ordinal.
I still have untouched pre-repair backups and can provide additional anonymized structural diagnostics if useful, but I would prefer not to upload real rollouts because they contain private conversation/project data.
x1507078102-commits commentedon Sep 12, 2026
Additional real-world evidence from Codex desktop.
Environment
Persisted delegation threads remained visible in the sidebar, but opening them showed blank history.
Local inspection found 13 affected threads with
source: "vscode"andthread_source: "subagent". Twelve hadsubagent_history_start_ordinalequal to the rollout record count. The remaining thread had only two later settings events after that boundary. Existing message records were therefore excluded from the displayed history.Four affected threads were unarchived conversations still in use. After backing them up, correcting their history boundaries, rebuilding the derived history indexes, and reloading them, the history APIs returned 3, 3, 7, and 5 turns respectively. The message-bearing rollout bodies after the metadata line remained byte-for-byte unchanged during recovery. No replacement conversation content was generated.
This appears to match #38762. Please consider a regression test for a persisted, named thread with
source=vscode,thread_source=subagent, and its own completed turns, plus a safe repair path for existing affected threads.Limitations
No transcripts, logs, local paths, real thread IDs, or account identifiers are included in this comment.