Codex Desktop 0.153.4: existing task loses access to days of history after reopen; persisted projection cursor points behind expected ordinal #43182

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   appIssues related to the Codex desktop appIssues related to the Codex desktop app

   sessionIssues involving session (thread) management, resuming, forking, naming, archivingIssues involving session (thread) management, resuming, forking, naming, archiving

   on Sep 6, 2026
2. github-actions commented on Sep 6, 2026

   

   github-actionsbot

   on Sep 6, 2026 – with GitHub Actions

   Contributor

   More actions

   Potential duplicates detected. Please review them and close your issue if it is a duplicate.

   • macOS desktop history freezes at older turns: projection expected ordinal 3185, got 3184; migration says already_paginated #43124
   • [0.153.4][Windows] Resume reuses two rollout ordinals after trailing token_count records, freezing desktop history #43142
   • Windows 26.901.4073.0 / CLI 0.153.1: duplicate rollout ordinal stalls history; later conversations disappear after restart #42999
   • macOS: newest-first thread lookup returns two-day-old in-progress turn despite newer persisted messages #43129
   • [Windows][26.831.1445.0] App update/resume duplicates a rollout ordinal and hides later task history #42157

   Powered by Codex Action
3. ptrgonow commented on Sep 6, 2026

   

   ptrgonow

   on Sep 6, 2026

   Author

   More actions

   Follow-up to the automated duplicate suggestions, checked on September 6, 2026 (UTC).

   The affected task in this issue is still not recovered on the installed 0.153.4 runtime. There are relevant merged upstream fixes, but the public rust-v0.153.4 source still lacks the relevant changes.

   Review of all five suggested duplicates

   I read each issue's body, all returned comments, and timeline. All five are currently open.

   Report	Relationship to this case	Resolution evidence
   #43124	Closest match: local macOS, bundled 0.153.4, stale newest-first history, checkpoint expecting N while the next record contains N-1.	Diagnostic discussion and proposed repairs; no verified product recovery.
   #43142	Windows 0.153.4; multiple reused ordinals after trailing token_count events.	A comment identifies #42378; another reports recovery on an isolated, ordinal-corrected copy. Neither establishes recovery of this unmodified task.
   #42999	Windows 0.153.1; a duplicate settings-event ordinal stalls a linked history segment.	Reporter verified a guarded manual projection-cursor workaround, including a second cold backend read. Desktop refresh confirmation was explicitly pending.
   #43129	macOS; newest-first lookup returns an old unfinished turn despite newer durable messages.	Similar independently observed retrieval failure, but its original report did not inspect projection ordinals or establish a repair.
   #42157	Windows; settings event reuses the preceding token-count ordinal, matching our newly rechecked boundary below.	Reporter recovered 19/19 user messages after manually advancing a backed-up projection cursor. This is case-specific recovery, not a shipped product fix.

   These are strong candidates for consolidation under the same projection-stall family, especially #43124 and #42157. That does not establish an identical initiating cause in every report. The related closed #42245 has stateReason=DUPLICATE, which is not evidence of a fix.

   Relevant upstream fixes and the release distinction

   • Keep SQLite history projection moving past invalid records #42369, merged September 2 at 095ac4f131e759b204fa6368dc42d2feff6eb21a, changes the materializer to advance past duplicate/regressed ordinals and other invalid records. Its regression test covers a duplicate ordinal followed by later valid history. Current main retains this handling and reads from the persisted byte/ordinal checkpoint, so it is relevant to already-stalled suffixes, not only prevention.
   • Route rollout reads through the canonical JSON decoder #42378, merged September 2 at 69cebb5d15939bf9b6c1b4647b53879beab91ba2, routes the reverse ordinal scanner through the canonical rollout decoder. Its added recorder test uses one decimal-valued token-count record and checks continued ordinal sequencing after resume. It does not independently demonstrate recovery of an already-stalled existing task or the exact multiple-token-count case in [0.153.4][Windows] Resume reuses two rollout ordinals after trailing token_count records, freezing desktop history #43142.

   I checked the release source directly, not just the version number or a comment:

   • rust-v0.153.4 ordinal scanner still calls scanner.scan_next::<RolloutLine>() and continues past rejected records.
   • rust-v0.153.4 materializer still returns an error when ordinal < next_ordinal.
   • GitHub's compare API reports the release tag as diverged from both merge commits. Direct source inspection also rules out those specific changes having been backported under different hashes.
   • The latest stable GitHub release returned at review time is still rust-v0.153.4. This does not establish what is available through every desktop update channel or prerelease channel.

   Consequently, “fixed on main” and “recovered in the currently installed desktop release” are different claims. The PRs contain regression tests, but I did not execute those Rust tests. Their reported CI rollups include failures as well as successes; I am not claiming all CI passed or attributing those failures to these changes.

   Fresh read-only verification of this affected task

   At approximately 2026-09-06 08:15 UTC:

   • Desktop bundle remains 26.901.51231, build 8109; the bundled executable still reports codex-cli 0.153.4. The latest located local app-server version log reports 0.153.4 at 07:53:04.047Z.
   • A cursor-free read_thread request with turnLimit: 1 still returns the September 2 turn (startedAt=1788343817, completedAt=null) as newest, with hasMore=true. The fresh tool response calls its status interrupted; the SQLite row still says inProgress. This updates the earlier tool-status observation without changing the stale-history finding.
   • The projection remains at byte 18,567,836, expected ordinal 3500. The newest projected turn remains ordinal 3308.
   • A full scan of a 320,572,371-byte / 20,950-record rollout snapshot found zero JSON parsing failures and exactly one top-level ordinal discontinuity:

   Physical line (1-based)	Byte offset	Event	Ordinal
   3500	18,566,946	token_count	3499
   3501	18,567,836	thread_settings_applied	3499, reused
   3502	18,571,205	task_started	3500

   • Later actual records include a user message at 2026-09-06T08:14:19.673Z and task_complete at 2026-09-06T08:14:49.212Z (ordinal 20948). These are parsed record metadata, not timestamps quoted inside transcript text.

   The local failure boundary is therefore confirmed more precisely than in the original report: a duplicate settings-event ordinal immediately after a token-count event, with the persisted checkpoint still pointing at that duplicate. This matches the strict rejection in the public 0.153.4 materializer. It does not by itself prove why the writer reused the ordinal.

   What remains to establish resolution

   Please identify the desktop/runtime release carrying both applicable fixes and verify the same existing affected task catches up, exposes the later saved messages, and remains correct after another restart. A source-level skip policy also needs recovery validation for conversation-bearing duplicate records; merely advancing a checkpoint is not proof of lossless history restoration.

   I am leaving this issue open because recovery remains unverified and the stale-history failure still reproduces in the current installation. Consolidation with a canonical report is welcome, provided the applicable release and existing-task recovery are tracked explicitly.

   This follow-up used read-only local inspection and history reads. No app restart, rollout/database/cache modification, manual recovery, runtime replacement, or model test turn was performed. No raw conversation, private identifiers, credentials, or local paths are included.
4. argszero commented on Sep 6, 2026

   

   argszero

   on Sep 6, 2026

   More actions

   Excellent forensic work — the single-duplicate boundary at line 3501 is precisely the shape the current-main code is built to advance past. Adding the release-level facts and the code-level answer to your two open questions.

   Which release carries both fixes (verified against commit ancestry)

   • rust-v0.153.4 (still the latest stable, published 09-04): contains neither commit. Your tag-source inspection is correct — ordinal.rs there still calls scan_next::<RolloutLine>() and the materializer still errors on ordinal < next_ordinal.
   • rust-v0.154.0-alpha.3 (published 09-04): contains both 69cebb5d15 (Route rollout reads through the canonical JSON decoder #42378) and 095ac4f131 (Keep SQLite history projection moving past invalid records #42369) — I verified both hashes are ancestors of that tag. So the 0.154.0 line is the first to carry them; the next stable release after 0.153.4 will include both.

   Would your exact task catch up on a fixed build? (code answer)

   Yes, for this boundary — the current-main batch projector handles ordinal < next_ordinal by skipping and continuing, not erroring (thread_history_materialization.rs):

   if ordinal < next_ordinal {
       warn!(..., "skipping duplicate or regressed rollout ordinal during projection");
       record_projection_anomaly(ProjectionAnomaly::DuplicateOrRegressedOrdinal);
       next_offset = line_end_offset;
       line_start_offset = line_end_offset;
       continue;
   }

   Applied to your numbers: checkpoint next_rollout_ordinal = 3500 at byte 18,567,836; the line there carries the duplicate 3499 → skipped (byte checkpoint advanced, anomaly recorded); the next line (18,571,205) carries ordinal 3500 = expected → projection continues normally from there to your ordinal-20948 tail. That is exactly the scenario #42369's regression test locks in (a duplicate ordinal followed by later valid history), and it is why the materializer reads from the persisted byte/ordinal checkpoint rather than re-scanning.

   Writer-side cause of the duplicate: a reused top-level ordinal is what the pre-#42378 resume path produced — ordinal_state_for_rollout mis-derived a stale next ordinal after a decimal-bearing token_count tail (the #43142 mechanism), and the next append (thread_settings_applied) reused it. Your zero-parse-failure / exactly-one-discontinuity scan is consistent with that: the writer never corrupted the line, it just picked an already-used ordinal. #42378 removes that failure mode, so a fixed build both stops creating new duplicates and advances past the existing one.

   Your lossless-skip caveat, scoped

   Agreed, and it matters here: the skipped record at line 3501 is thread_settings_applied — settings metadata, not conversation content — so skipping it is conversation-lossless in this task. The general policy question stands for a hypothetical conversation-bearing duplicate (skipping would silently drop a message), and that is where the current code has no answer: the strict apply_projection path (thread_history.rs, used by thread/revert) and migrate-rollouts (returns AlreadyPaginated without checking projection consistency) still have no repair path for an already-stalled task. I outlined that gap and two fix directions on #43124 — a per-task consistency-check/recovery command would live there, not in the decoder/materializer fixes.

   Suggested verification on your side

   Reopen the same task on a 0.154.0-alpha.3+ (or next-stable) build and confirm: projection advances past byte 18,567,836, the Sept 2–6 turns appear in newest-first reads, and a subsequent restart stays correct. If it doesn't advance (i.e., the resume path drives the strict apply_projection route rather than the tolerant batch projector), that is the #43124-family gap — separate from these two fixes — and worth reporting there with your byte-level evidence.
5. argszero commented on Sep 6, 2026

   

   argszero

   on Sep 6, 2026

   More actions

   Your open question — "verify the same existing affected task catches up on a release carrying both fixes" — now has an empirical answer, from an independent Windows reporter on #43262.

   #43262: same family, successful 0.154.0-alpha.3 catch-up

   CanberkGencay (Windows desktop, stable 0.153.4, 608 MB / ~92k-record rollout, thread running Aug 29 → Sep 6) recovered the full history with exactly the release sequence predicted above:

   • 0.153.4 aborts at the duplicates. After he manually repaired an initial cursor desync, stable 0.153.4 scanned to the first duplicated ordinal and stopped: expected=21964 got=21963 from live_writer — the same expected ordinal N, got N−1 family error (it is the strict pre-Keep SQLite history projection moving past invalid records #42369 materializer; no skip).
   • 0.154.0-alpha.3 rebuilt the entire projection. Resume under the alpha "continued through both duplicated ordinal locations instead of aborting": projection byte offset → 608,475,078 (bytes remaining 0), 27,857 thread items, 42 turns — full history restored in the desktop app afterward.

   So: on a build containing #42369 (skip regressed ordinals) + #42378 (canonical decoder), an already-stalled task does catch up on its own. That confirms the batch-projector skip path I quoted in my previous comment handles your exact boundary shape.

   His duplicate boundaries match the writer-side pattern

   His two duplicate ordinals both sit immediately after a token_count record:

   • ordinal 21963 event_msg token_count → ordinal 21963 event_msg task_started → 21964 world_state
   • ordinal 87104 event_msg token_count → ordinal 87104 event_msg thread_settings_applied → 87105 thread_settings_applied

   That is the same "decimal-bearing token_count tail poisons the pre-#42378 resume ordinal scan, next append reuses" mechanism (also now corroborated by a Linux/WSL2 report on #43124 with cursors parked at thread_settings_applied, and your own duplicate thread_settings_applied at line 3501).

   One new datapoint worth tracking: the initial desync ran the other way

   His pre-existing projection state was (byte offset 122207, ordinal 16) while the line at 122207 carried ordinal 17 — i.e., the byte cursor was one line ahead of the ordinal (expected ordinal 16, got 17), the inverse of your expected 3500, got 3499 shape where the ordinal is ahead of the line. Both directions have now been observed across the family (#43124: ordinal ahead; #43262: byte cursor ahead). Since thread_history.rs writes both fields in a single transactional upsert, a diverged pair implies the two frontiers were committed separately somewhere in the pipeline (likely the live-writer append path) — that is a distinct, testable writer-side hypothesis from the duplicate-ordinal mechanism, and worth a look if maintainers want to close the family at the source.

   Net: your task should catch up on 0.154.0+ exactly as #43262 demonstrates; if it does not, that would isolate a strict-apply_projection-only path that the batch projector doesn't cover, which is the #43124 repair-gap scenario.
6. iamhectorlopez commented on Sep 6, 2026

   

   iamhectorlopez

   on Sep 6, 2026 – with ChatGPT Codex Connector

   More actions

   Additional local macOS occurrence on September 6, 2026, with the same persisted checkpoint / repeated-ordinal pattern. An additional user-visible symptom is that Cmd+F returns 0 results for saved messages that are missing from the displayed conversation.

   Environment

   • Desktop bundle: 26.901.51231, build 8109, read from Info.plist.
   • Bundled executable: codex-cli 0.153.4, verified with --version; the running app-server version was not separately verified.
   • macOS 26.6.2 (25G83), local desktop task.

   Observed behavior

   1. Open the affected existing task and search for an earlier sent message with Cmd+F.
   2. The full message and multiple shorter phrases all return 0 results. Screenshots confirm the correct task and the zero-result searches.
   3. The missing exchange is present in the task's local rollout as actual user/assistant messages, including completed UserMessage events. No rollback event was found removing it.
   4. Current work can still appear in the live conversation while the older exchange remains inaccessible. The task-status tool also returns an older inProgress turn despite later durable messages.

   The user reports that this behavior did not occur previously. The last known-good app version and initiating event are unknown; a full-app-restart recovery test has not been verified.

   Read-only diagnostic evidence

   The affected task's projection state is:

   next_rollout_byte_offset = 205640498
   next_rollout_ordinal     = 5001
   

   The surrounding actual records are:

   Byte offset	Event	Stored ordinal
   205639599	token_count	5000
   205640498	thread_settings_applied	5000, repeated
   205643930	task_started	5001

   Thus the stored checkpoint expects ordinal 5001, but points exactly at the settings event carrying 5000.

   The newest projected turn remains at ordinal 4809, status inProgress, with no completion timestamp. A scan through 891,554,298 bytes / 12,202 JSONL records parsed successfully and found exactly one top-level ordinal discontinuity, the repeated ordinal above; later saved records extended through ordinal 12200.

   This corroborates the stalled-history pattern in this issue. It does not establish a separate Find defect or prove why the ordinal was reused.

   Impact: earlier requirements, decisions, and approvals become difficult to recover through the product, making saved messages appear lost. Please include recovery and searchability of the same affected existing task when validating a fix.

   All diagnostic inspection was read-only. No original rollout, database, or cache was modified. Private message text, task identifiers, project names, screenshots, and raw logs are omitted.
7. argszero commented on Sep 6, 2026

   

   argszero

   on Sep 6, 2026

   More actions

   This is the fifth independent datapoint with the exact same boundary shape, and it's byte-for-byte consistent with the pattern already documented on this thread:

   • checkpoint next_rollout_ordinal = 5001 parked at byte 205,640,498;
   • the line there is a thread_settings_applied carrying the repeated ordinal 5000 (right after the token_count at 5000), and the next line (task_started) carries 5001.

   That is the same "stalled cursor at a repeated settings-event ordinal immediately following a token-count record" shape as ptrgonow's line-3501 duplicate on this thread, CanberkGencay's second duplicate (ordinal 87104) on #43262, and the WSL2 cursors on #43124. On the 0.153.4 build you're running, the strict materializer stops at that repeated ordinal, so everything after it (through ordinal 12200) never gets projected — which is why the newest projected turn sits at 4809 and the later exchange is invisible.

   On Cmd+F returning 0 results: that is a consequence of the projection stall, not a separate find defect. The UI and its search operate over the materialized conversation (the projected thread turns/items), not the raw rollout. Messages that were never projected are absent from both the rendered transcript and the search index, even though they exist intact in the JSONL — matching your observation that the exchange is present as real user/assistant messages in the rollout with no rollback event.

   Recovery: the empirical fix path is confirmed on this family — #43262's reporter recovered a stalled 608 MB task by resuming under 0.154.0-alpha.3 (which contains #42369's tolerant materializer: lines with ordinal < next_ordinal are skipped with a recorded anomaly instead of aborting). Your boundary (line at the checkpoint carrying 5000 while the state expects 5001) is exactly that shape, so the same resume-under-0.154.0+ should advance the projection to the true end of the 891 MB rollout, after which both the history and its searchability should return for that same affected task. If it does not catch up, that isolates a strict-apply_projection-only path that the batch projector doesn't cover — which is the open repair-gap question on #43124/#43182.

   Agreed that recovery and searchability of the same affected existing task should be the validation criteria — that is precisely the open verification item (ptrgonow's ask on this thread, now empirically answered once by #43262, and worth confirming on a second stalled task).
8. argszero commented on Sep 6, 2026

   

   argszero

   on Sep 6, 2026

   More actions

   One correction to my previous comment on this thread: I floated the hypothesis that the reverse-direction desync (#43262's initial (byte offset 122207, ordinal 16) at an ordinal-17 line) implied the two projection-state fields were committed separately in the live-writer append path. Reading the code, that is not the case on current main: the only production writer of thread_history_projection_state is a single transactional upsert in thread_history.rs that sets next_rollout_byte_offset and next_rollout_ordinal together, at the end of each apply_projection batch. There is no path on current main that commits one field without the other.

   What the code does show is that the tolerant reader handles both desync directions:

   • ordinal < next_ordinal (duplicate/regressed — the shape on this thread and [0.153.4][Windows] Resume reuses two rollout ordinals after trailing token_count records, freezing desktop history #43142/I recovered a long-running Codex thread with missing history | Chat History is not beeing Displayed #43262's second boundary) → the line is skipped, the byte checkpoint advances, anomaly DuplicateOrRegressedOrdinal is recorded (thread_history_materialization.rs);
   • ordinal > next_ordinal (forward gap — the shape of I recovered a long-running Codex thread with missing history | Chat History is not beeing Displayed #43262's initial (122207, 16) state) → a SkippedOrdinalRange is recorded and projection continues from the higher ordinal (anomaly ForwardOrdinalGap).

   Both are pre-#42369 strict-reader failure modes; both advance on a 0.154.0+ build. So the "separate field commits" hypothesis should be discarded — the observed desyncs are read-side strictness artifacts (and, for the ordinal-reuse shape, writer-side pre-#42378 resume ordinal derivation), not a current-main projection-state atomicity bug. The only genuinely open repair gap remains the strict apply_projection path used by thread/revert and the already_paginated short-circuit in migrate-rollouts (#43124), which no released build yet addresses.
9. dahutos2 commented on Sep 9, 2026

   

   dahutos2

   on Sep 9, 2026

   Last edited by dahutos2

   More actions

   Additional macOS occurrence of the duplicate-ordinal pattern already documented here. I have read the discussion of #42369, #42378, and the reported 0.154.0-alpha.3 recovery. This report does not test those fixes; it adds a stop/restart timeline from an unmodified affected thread.

   Environment: Desktop 26.901.51231 (build 8109), bundled and Desktop-reported app-server 0.153.4, macOS 26.2 (25C56), arm64.

   Observed sequence on September 3 (UTC):

   • 02:41:57.613: token_count persisted with ordinal 6293.
   • 02:42:04.986: Desktop stopped the app-server transport.
   • 02:42:12.167: the restarted app-server reported 0.153.0-alpha.5.
   • 02:43:01.402: thread_settings_applied reused ordinal 6293.
   • 02:43:01.404: task_started used ordinal 6294.

   The persisted history-projection cursor points exactly at the repeated settings record:

   next_rollout_byte_offset = 40625817
   next_rollout_ordinal     = 6294
   actual ordinal at that offset = 6293
   

   Backend warnings repeatedly report:

   thread history projection for <redacted>
   expected ordinal 6294, got 6293
   

   A full scan of 48,410,495 bytes / 8,046 JSONL records found zero JSON syntax failures and exactly one ordinal discontinuity. Twelve subsequent turns have matching task_started/task_complete records, extending through September 8, but remain beyond the stalled projection.

   After restarting Desktop on September 9 with app-server 0.153.4, resume again selected the old September 3 turn. The restart sequence is observed evidence, not proof of the writer-side cause.

   Recovery of this thread on a build containing the fixes has not been tested. I am adding this as corroborating evidence, not a separate regression or evidence that the fixes fail.

   All diagnostic inspection was read-only. No original rollout, database, cache, or configuration was modified; the rollout hash matched before and after inspection. Private content, identifiers, credentials, and paths are omitted.
10. ptrgonow commented on Sep 10, 2026

    

    ptrgonow

    on Sep 10, 2026

    Author

    More actions

    Thanks for the analysis, corrections, and additional reports. A status update from my side on September 10:

    My installed Desktop is now 26.903.61454 (build 8378), but both the bundled executable and today's running app-server log still report 0.153.4. The affected task's history projection remains stalled at the previously reported boundary. I am aware that the public rust-v0.154.0 release includes #42369 and #42378; I am waiting for a Desktop update that applies those fixes to my installation.

    Source review and earlier testing of an isolated copy give me confidence in the fix for this particular case: the duplicate is a thread_settings_applied record, and the isolated copy recovered under 0.154.0-alpha.3 and remained correct after a backend restart. That does not yet establish recovery in my actual Desktop installation. The original rollout and database have not been manually repaired.

    I will keep this issue open while waiting for the update. Once it is applied, I will verify the same existing affected task:

    • The projection catches up and previously missing conversations appear.
    • Previously missing messages can be found with in-thread search.
    • The recovered history remains available after a full Desktop restart.

    If those checks pass, I will post the Desktop/runtime versions and verification results here, then close this issue. If they fail, I will add the updated diagnostic evidence instead.

    ❤️React with ❤️1Reacted by Daichi Okumura
11. ptrgonow commented on Sep 14, 2026

    

    ptrgonow

    on Sep 14, 2026

    Author

    More actions

    Recovery confirmed for the original affected task on September 14. Closing this report as resolved for my case. Thanks to everyone who investigated and contributed corroborating reports.

    Installed Desktop: 26.908.40834, build 8881. Bundled and running app-server: 0.154.0-alpha.6.2, verified from the executable and today's startup log.

    The previously missing history now appears restored in Desktop. A fresh history read returned the completed September 11 turn instead of the stale September 2 turn. Further work has since continued in this same task.

    I also performed a read-only comparison of a consistent SQLite snapshot against its corresponding original JSONL prefix:

    Check	Result
    Projection byte offset	462,181,720, matching the rollout EOF at verification time
    Next ordinal	27582 in both the projection and raw prefix
    Turns	331 raw started turns / 331 projected turns; none missing
    Terminal turns	328; no mismatches in checked terminal status/completion timestamps
    Canonical items	8,870 raw unique items / 8,870 projected items; no missing or extra items
    Message content	401 user messages and 1,016 agent messages matched after normalizing representation differences
    Original duplicate	Still present: settings event ordinal 3499 at byte 18,567,836; later history is now projected

    The original rollout, database, and configuration were not manually repaired. The duplicate settings line remains in the original file, so this recovery did not require deleting or renumbering it.

    Startup logs show separate Desktop launches on September 12, 13, and 14 with the updated runtime. The same task was resumed/read on September 12 and again on September 14; today's backend read successfully returned the newer persisted history. I did not force an additional app restart during this verification. Cmd+F was not separately exercised, and image file contents/rendering were not validated.

    This closes the original stale-history/projection-stall report based on the recovered history and raw-to-projected message checks. It is not a claim that skipping arbitrary conversation-bearing duplicate ordinals is generally lossless. If the original symptom recurs, I will follow up with the affected version and diagnostic evidence.

    👍React with 👍1Reacted by Daichi Okumura
12. ptrgonow

    closed this as completedon Sep 14, 2026