Codex paginated rollout can emit duplicate ordinal after unfinished turn, permanently freezing thread history projection

What version of the Codex App are you using (From “About Codex” dialog)?

The latest version as of 29th August 2026

What subscription do you have?

ChatGPT Plus

What platform is your computer?

Windows 10 latest 64 bit enrolled in extended security updates running intel lga1150 platform

What issue are you seeing?

A long-running Codex thread became permanently stuck displaying an old version of its conversation history even though its canonical paginated rollout JSONL continued to receive hundreds of MB of later events.
Forensic inspection shows that the canonical rollout itself contains a duplicate top-level ordinal at the exact byte where thread_history_1.sqlite stops materializing history.
At the first failure boundary:
Physical line 9814
ordinal: 9813
type: token_count
byte end: 143293415

Physical line 9815
ordinal: 9813 <-- duplicate
type: thread_settings_applied

Physical line 9816
ordinal: 9814
type: task_started
The thread-history projection cursor stops exactly after physical line 9814:
next_rollout_ordinal = 9814
rollout byte offset = 143293415
The materializer subsequently reports:
expected ordinal 9814, got 9813
This causes the derived thread history to remain permanently frozen even though the canonical rollout continues.
A second duplicate-ordinal discontinuity was found later in the same rollout at physical line 12831. Both discontinuities have the same observed pattern: an unfinished turn without task_complete, followed by token_count, then a later thread_settings_applied that repeats the preceding ordinal.
Restoring the original thread_history_1.sqlite restores the UI exactly to the stale cutoff. Removing the projection database results in an empty history database; normal startup does not replay the existing paginated rollout into it.
The supported migration classifier was also tested against an isolated copy:
{
"thread_id": "01a03a72-2f06-70b0-a3b8-a309fe1ba833",
"status": "already_paginated",
"bytes_processed": 0,
"message": null
}
Therefore legacy rollout migration does not repair the affected already-paginated thread.

What steps can reproduce the bug?

I do not yet have a safe deterministic minimal reproduction, but both observed failures share this sequence:
Have a paginated Codex thread with an active root turn.
The turn ends abnormally or otherwise remains unfinished without a task_complete event.
The final event associated with the turn includes a token_count.
Later begin another task in the same thread.
In the affected cases, the emitted thread_settings_applied record repeats the preceding record's top-level ordinal instead of incrementing it.
task_started then uses the ordinal that thread_settings_applied should have used.
The history materializer reaches the duplicate and rejects it with expected ordinal N+1, got N.
The canonical rollout continues growing, but the rendered/history projection remains permanently frozen at the last valid record.
I found this pattern twice in the same affected rollout. Normal completed turns immediately before the first failure instead follow:
token_count
→ task_complete
→ thread_settings_applied (next ordinal)
→ task_started (next ordinal)
I have deliberately not attempted to manufacture an unfinished turn by killing Codex because I cannot establish that forced termination is equivalent to the original failure mode.

What is the expected behavior?

Every top-level record in a paginated rollout should receive a unique monotonically increasing ordinal.
Starting a new task after an interrupted or unfinished previous turn should not cause thread_settings_applied to reuse the preceding event's ordinal.
If an incomplete turn is encountered, Codex should either recover/close that turn or otherwise continue writing a valid event stream that the thread-history materializer can consume.
A malformed event should also ideally not make all subsequent valid conversation history permanently inaccessible in the UI.

Additional information

Feedback was also submitted through the Codex app.
Feedback ID: 01a04dc0-1ee7-7b52-9fd9-7a49c827d2f8
Additional forensic observations:
Affected rollout is already history_mode = paginated.
Canonical rollout remained intact and continued well beyond the frozen projection.
First duplicate occurs immediately after the projection's final accepted byte.
A second duplicate ordinal occurs later in the same rollout with the same unfinished-turn transition pattern.
Three normal completed turns immediately preceding the first failure contain task_complete and have consecutive ordinals.
thread_history_1.sqlite accurately represents history only through the last valid ordinal before the discontinuity.
Removing the history projection DB causes Codex to create an empty replacement rather than replaying the existing paginated rollout.
codex migrate-rollouts --json --thread on an isolated copy returns already_paginated with bytes_processed: 0.
I have preserved the original affected rollout and projection databases and can provide additional sanitized logs, exact event windows, byte offsets, or database state if useful.
No canonical rollout data was modified during diagnosis.

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

[Desktop][Windows] Paginated thread history stalls on duplicate ordinal while rollout remains complete #41079
[Bug] Competing paginated rollout writers emit overlapping ordinal sequences #41353

Powered by Codex Action

Related to issue #41079 possibly and issue #41353 appears to describe the same underlying ordinal-writer failure class. My affected rollout provides an additional concrete instance where thread_settings_applied repeats the preceding token_count ordinal, permanently wedging history materialization. I also found a second duplicate-ordinal transition later in the same rollout. The missing task_complete correlation may be incidental to the broader stale/competing-writer mechanism described there.

Additional confirmed instance on macOS Desktop with iOS Remote symptoms.

Environment:

ChatGPT/Codex macOS app: 26.825.41651
Embedded CLI: 0.151.0-alpha.7.1; standalone CLI used for verification: 0.151.0
macOS 27.0, Apple Silicon

User-visible impact:

A long-lived thread showed older conversation state, missing/out-of-order turns, and intermittent non-response in the macOS app.
The same thread appeared inconsistent through iOS Remote, making the failure look like cross-device sync corruption.
A native same-directory fork failed with: thread-store internal error: thread history projection expected ordinal 4516, got 4515.

Forensics:

The canonical paginated rollout was approximately 37 MB and contained exactly one duplicate top-level ordinal at the failure boundary.
The repeated ordinal was 4515; the duplicate record was thread_settings_applied at a resume boundary.
The projection state expected ordinal 4516 at the byte offset where the second 4515 appeared, so materialization could not advance even though the canonical rollout continued.
All records before this boundary were contiguous.

Verified recovery (after preserving byte-for-byte backups):

Increment the duplicate record and every subsequent top-level ordinal by one.
Validate the repaired rollout as a continuous sequence (0..5280, 5,281 ordinal-bearing records, zero anomalies).
Remove only this thread's generated history-projection rows and let the app server rebuild them.

After that repair, the same-directory fork succeeded and the repaired child thread completed a validation turn normally. The original affected thread was archived rather than deleted. A post-restart codex doctor run reported 22 checks OK, 0 warnings, and 0 failures.

This case supports the same underlying failure described here: a resumed writer can reuse the preceding ordinal, and the materializer does not self-heal. It also shows that the resulting local projection failure can surface to users as an apparent macOS/iOS Remote sync problem. Ideally ordinal allocation should be serialized and existing malformed projections should be detected and rebuilt or repaired without requiring direct rollout/database intervention.

I retained the original rollout and pre-repair projection database locally and can provide a narrowly sanitized event window or checksums if maintainers need them.

Codex paginated-history producer and self-healing projection defect

Sanitized evidence

This report contains aggregate structure only. It excludes thread identifiers, prompts,
conversation content, local paths, credentials, and raw logs.

Runtime observed: Codex Desktop signed runtime 0.151.0-alpha.7.1.
Catalog audited: 2,851 paginated threads.
Inconsistent projections: 20.
Producer discontinuities: 9 adjacent token_count(N) / task_started(N) pairs and
2 duplicated no-op thread_settings_applied(N) records.
Derived cursor faults: 5 cursors inside canonical records and 4 cursors beyond a
migration-rewritten canonical EOF.
Representative runtime error: expected ordinal 1073, got 946.
Canonical rollout and catalog databases remained authoritative and were not edited.

The failure is not renderer-only. The history projector stops at the last valid SQLite
prefix and later reads keep returning that prefix even though the canonical rollout grows.

Tested reference patch

The attached patch applies to openai/codex main commit
63d213884daea50e4f74efc192cdc44f549b67d5.

It implements these invariants:

A rollout writer that reopens after an ambiguous failure reopens the current pathname,
not a stale inode, and recomputes its ordinal allocator from the last complete durable
record before retrying buffered items.
Compatibility metadata writes use the same per-thread live-writer lock and recorder as
lifecycle writes. An unloaded thread additionally uses the writer coordinator.
Incremental projection remains strict. If an existing cursor is stale, mid-record, or
out of range, the projector parses the complete canonical rollout before touching SQLite.
Strict rebuild consumes a duplicated no-op settings record. It also replaces only an
immediately preceding history-no-op token_count(N) projection step with the meaningful
adjacent task_started(N) record.
Meaningful-versus-meaningful ordinal conflicts, malformed JSON, unknown records, and
incomplete durable tails remain fatal.
Candidate rows replace only the affected thread's derived history rows inside one SQLite
transaction. Readers keep the old readable prefix until commit; any error rolls back.

Regression results

codex-rollout: writer reopen recomputes ordinal state from the durable tail — passed.
codex-thread-store: duplicate token-count/task-start strict rebuild — passed.
codex-thread-store: stale cursor rebuild is atomic and malformed canonical history
preserves the old prefix — passed.
Existing invalid-complete-suffix rejection test — passed.
Metadata update compatibility suite — 24 passed.
cargo fmt --all -- --check — passed (stable rustfmt only emitted the repository's
existing nightly-option warnings).

App-server and renderer integration design

The thread store should return a typed projection outcome rather than only logging:

up_to_date
repaired { old_cursor, new_cursor }
unrecoverable { cursor, canonical_fault_class }

On repaired, the app server should invalidate the thread-history cache and emit a
thread-scoped history/projectionRepaired notification. The desktop client should refetch
the affected thread/turn list while preserving selection and scroll anchoring.

On unrecoverable, list/read responses must carry a visible degraded-history status. The
desktop client should render a persistent error affordance with a retry action rather than
silently displaying the last valid prefix as if it were current. No automatic path may skip
meaningful conflicts.

Materialization should run on append/flush and as a bounded read-path preflight whenever a
stored cursor is not an exact canonical record boundary or EOF. Successful rebuild must be
committed before the repaired notification is emitted.

Release acceptance

A signed release is complete only when it:

prevents new duplicate/regressed ordinals under interrupted/resumed and concurrent-metadata
fixtures;
automatically repairs the retained stale, midline, and EOF-rewrite fixtures;
exposes unrecoverable projection faults in the desktop UI;
passes signed native-reader and real desktop UI canaries after interruption/resume; and
produces no new failed to project durable rollout warnings in a fresh follow-up cycle.

Until then, local derived-state repair is mitigation only; upstream remains pending.

Disclosure: this comment was generated by Codex from local diagnostics and posted with the user's explicit approval. It contains no prompt/conversation content, credentials, local paths, or thread identifier.

Another confirmed macOS instance, on a slightly newer signed desktop runtime:

ChatGPT/Codex macOS app: 26.825.51511
Embedded Codex CLI/core: 0.151.0-alpha.7.2
macOS 26.6.2 build 25G83, Apple Silicon (arm64)

User-visible impact:

The desktop transcript stopped at an older turn while later user inputs continued reaching the live model context.
The persisted task API exposed only the frozen history prefix.
A same-directory fork failed repeatedly with thread-store internal error: thread history projection ... expected ordinal 1256, got 1255.

Forensics:

The canonical paginated rollout was 37.8 MB / 6,586 physical lines at initial inspection.
A complete ordinal scan found exactly one discontinuity:
- physical line 1256: ordinal 1255, token_count
- physical line 1257: ordinal 1255, duplicate thread_settings_applied
- physical line 1258: ordinal 1256, task_started
thread_history_projection_state stopped at byte offset 9542592, expected ordinal 1256; that offset was exactly the end of the last valid line and start of the duplicate.
Local diagnostics recorded 345 repetitions of the ordinal error over roughly 24 minutes.
The bundled dry-run migration reported already_paginated, bytes_processed: 0.

Verified local recovery, after preserving the rollout, projection database, and exact malformed line:

Atomically advance only this thread's projection checkpoint from byte offset 9542592 to 9545643, keeping expected ordinal 1256, thereby skipping the duplicate metadata line.
The live writer then replayed the retained suffix successfully; the task API advanced from the stale prefix to 43 turns and 2,248 projected items, and paginated thread reads worked again.
For durable canonical repair without shifting any stored byte offsets, replace the 3,051-byte duplicate record with a same-length whitespace line. The materializer explicitly consumes all-whitespace lines without consuming an ordinal.
Re-scan the full canonical rollout: zero remaining ordinal gaps or duplicates. Projection continued advancing normally.

This was a narrowly targeted, backed-up local recovery, not a recommendation for an unsupported end-user repair flow. The case independently confirms the same token_count(N) -> thread_settings_applied(N) -> task_started(N+1) producer defect on macOS and shows that a no-op duplicate can be skipped while preserving later offsets. The upstream fix should still prevent duplicate allocation and make the projection self-heal.

I can confirm the same failure on Linux x86_64. The affected thread was created with Codex CLI 0.149.1 and diagnosed/recovered with 0.151.0.

The raw rollout remained valid JSON and contained 51,450 lines (537,124,417 bytes), but had exactly one duplicated top-level ordinal at zero-based physical line 2,379:

line 2,378: token_count, ordinal 2,378
line 2,379: thread_settings_applied, ordinal 2,378 (duplicate)
line 2,380: task_started, ordinal 2,379

The projection state stopped at ordinal 2,379 and byte offset 5,165,166, exactly where the duplicate began. Logs repeatedly reported expected ordinal 2379, got 2378. The raw rollout continued growing, but 49,071 subsequent lines were inaccessible through thread history. migrate-rollouts reported already_paginated and processed zero bytes.

Recovery was verified after stopping the active writer and backing up the rollout and databases:

Incremented the duplicated ordinal and every later top-level ordinal by one, without changing event payloads or conversation content.
Resumed the thread through the 0.151.0 app server.

The projection then advanced from 2,379 to 51,450 lines, from 3 to 111 turns, and from 735 to 16,571 projected items. JSON parsing, ordinal validation, and SQLite quick_check all passed afterward.

This confirms the issue is cross-platform and can occur on the CLI path. The content was not deleted; a single duplicated ordinal permanently froze projection at a resume boundary.

No thread identifiers, local paths, or conversation content are included here. This diagnostic comment was prepared by Codex and is being posted with the user’s explicit approval.

I have an independent Windows reproduction of the same failure mode, with one useful variation: the duplicated second record is task_started, not thread_settings_applied.

Environment:

Originator: Codex Desktop
Runtime recorded in the session: codex-cli 0.151.0-alpha.7.2
history_mode = paginated

Sanitized rollout audit:

63,687,274 bytes
8,256 physical JSONL lines
0 invalid JSON lines
ordinal range: 0 through 8254
no missing ordinal values
exactly one duplicated ordinal

The failure boundary is:

physical line 5487: ordinal 5486, event_msg/token_count
  timestamp: 2026-08-31T17:30:09.034Z

physical line 5488: ordinal 5486, event_msg/task_started
  timestamp: 2026-09-01T00:34:53.278Z

physical line 5489: ordinal 5487, response_item/message

The Desktop log reports:

thread history projection expected ordinal 5487, got 5486

The UI/history projection remains permanently at the earlier boundary, while the canonical rollout continues with valid later messages and eventually reaches task_complete. Restarting and reopening the thread does not make the projection catch up.

The user logged out/switched account and later reopened the task during the multi-hour gap. That is correlation, not proof that account switching is the root cause. The stronger evidence is that a resumed/new task writer reused the final durable ordinal after a long interruption.

This broadens the suspected writer bug beyond thread_settings_applied: ordinal initialization/allocation during resume or restart may be shared across multiple event types, including task_started.

Suggested regression case:

persist token_count at ordinal N
simulate interrupted/unfinished turn and writer restart/resume
emit task_started
assert task_started ordinal == N + 1
append a user message
assert the history projector reaches the new message

For prevention, the resumed writer should determine its next ordinal from the locked durable rollout tail rather than stale in-memory state. For existing affected rollouts, a supported local repair/reindex path should recover the projection without requiring users to edit the canonical JSONL.

This also closely matches #41079. I have retained the original rollout and projection databases, but have deliberately not posted the thread ID, local paths, prompts, tool output, credentials, or raw transcript. I can provide additional sanitized ordinal/type/timestamp windows if maintainers need them.

Additional confirmed occurrence with the same boundary pattern.

Environment:

Windows 11 Pro 10.0.26200 x64
Codex Desktop 26.825.6671.0
Remote SSH app-server / CLI 0.151.0
Paginated rollout, 35.6 MB and 9,136 records

Sanitized failure boundary:

ordinal 664: response_item / custom_tool_call_output
ordinal 665: event_msg / token_count
ordinal 665: event_msg / thread_settings_applied (collaboration mode changed from Plan to Default)
ordinal 666: event_msg / task_started
thread_history projection cursor: expected ordinal 666 at the byte where the second 665 begins

The live task continued successfully in memory for roughly 23 more hours and the canonical rollout reached ordinal 9687, with all later messages intact. Restarting Codex Desktop did not create the malformed record; it exposed the existing projection stall when history was reconstructed. The parent task then rendered only the stale prefix, while direct navigation to the continuation segment produced “no rollout found for thread id …”. codex migrate-rollouts --thread ... --json --verbose classified both segments as already_paginated and did not repair them.

Full local backups were preserved. Recovery is using the safer workaround documented in #41079: advance only the derived projection cursor past the duplicate metadata record, leaving the canonical rollout unchanged.

No prompts, thread IDs, absolute paths, credentials, or raw logs are included here.

Additional confirmed occurrence on macOS

I can confirm the same duplicate-ordinal and frozen-projection failure pattern on macOS.

Environment:

macOS 26.5.2 (25F84)
Codex Desktop 26.825.51511 (build 7377)
Codex runtime recorded in the affected session metadata: 0.151.0-alpha.7.2
Affected rollout: history_mode = paginated

Observed evidence:

The canonical rollout remained valid JSON and continued growing to 483,112,126 bytes after the UI history froze.
The projection cursor stopped at next_rollout_byte_offset = 82,436,928 with next_rollout_ordinal = 17061.
The second record using ordinal = 17060 begins at exactly byte 82,436,928.
The sequence at the failure boundary is:
1. first 17060: event_msg / token_count
2. second 17060: event_msg / thread_settings_applied
3. 17061: event_msg / task_started
This was the only duplicated ordinal in the affected continuation rollout.
The last projected turn remained inProgress; later valid events continued in the canonical rollout but were never materialized into thread_history_1.sqlite or shown in the Desktop UI.

Preceding lifecycle evidence:

An active turn started at 2026-08-31 08:04:36Z.
The Desktop IPC connection reported EPIPE, and the app-server transport stopped at 08:05:07Z.
A new Desktop process resumed the same conversation at 08:05:25Z with the latest turn reported as interrupted.
Starting the next turn at 08:05:38Z emitted the duplicated thread_settings_applied ordinal.

An account/logout followed by account/login/start occurred about 55 minutes earlier and also caused an app-server IPC reset. This may be a contributing lifecycle condition, but the available evidence does not establish that the account switch directly caused the later duplicate ordinal.

No canonical rollout data, projection database, or application files were modified during diagnosis. I can provide sanitized exact event windows, byte offsets, or relevant lifecycle log excerpts if useful. Raw rollout files are not being uploaded because they contain private prompts, source code, and local paths.

Summary

On Windows, multiple local Codex threads showed older, incomplete conversation state after the app was restarted or after a thread was reopened. Completed assistant responses had previously been visible, but the UI later displayed an earlier partial response or an apparently unfinished turn.

Read-only inspection found two distinct failure modes:

Some older long-running threads contain a duplicate top-level ordinal in the canonical paginated rollout. Subsequent events remain in the JSONL, but the rendered/history view stops at an older boundary.
Other continuation threads have strictly increasing ordinals and intact final responses, but the desktop app logs a resumed completed turn as both latestTurnStatus=completed and markedStreaming=true. These threads can still visually roll back to an older state.

The issue was first observed after reopening the app on 2026-09-01 and recurred across several threads on 2026-09-02.

Environment

App package: OpenAI.Codex
App version: 26.825.6671.0
App architecture: x64
Local standalone Codex binary: codex-cli 0.149.1
OS: Windows 11 Home China, version 10.0.26200, build 26200, 64-bit
Locale: zh-CN
Task type: local desktop tasks across multiple unrelated projects
Auto-update environment at the time of the incident: CODEX_SPARKLE_ENABLED=false at both User and Machine scopes
AppX events during the incident window referenced the same package version 26.825.6671.0; no version transition was observed

Failure mode A: duplicate ordinal followed by stale/frozen history

Two originally affected long-running threads retained later raw events, but each contained one duplicate top-level ordinal at a transition from token_count to thread_settings_applied.

Sanitized evidence:

Thread A (ID redacted, rollout size about 168.5 MB)
physical line 7530: ordinal=7529, type=event_msg, payload.type=token_count
physical line 7531: ordinal=7529, type=event_msg, payload.type=thread_settings_applied  <-- duplicate
physical line 7532: ordinal=7530, type=event_msg, payload.type=task_started

Thread B (ID redacted, rollout size about 117.9 MB)
physical line 3494: ordinal=3844, type=event_msg, payload.type=token_count
physical line 3495: ordinal=3844, type=event_msg, payload.type=thread_settings_applied  <-- duplicate
physical line 3496: ordinal=3845, type=event_msg, payload.type=thread_settings_applied
physical line 3497: ordinal=3846, type=event_msg, payload.type=task_started

Two additional older threads showed the same transition pattern during the 2026-09-02 check:

Thread E (ID redacted, rollout size about 151.2 MB): duplicate ordinal 7122
token_count (ordinal 7122) -> thread_settings_applied (ordinal 7122) -> task_started (ordinal 7123)

Thread F (ID redacted, rollout size about 26.5 MB): duplicate ordinal 3321
token_count (ordinal 3321) -> thread_settings_applied (ordinal 3321) -> thread_settings_applied (ordinal 3322) -> task_started (ordinal 3323)

When these threads were reopened, the desktop log reconstructed them as interrupted and still streaming:

2026-09-01T01:03:20.359Z
event=maybe_resume_success conversationId=<REDACTED>
latestTurnStatus=interrupted markedStreaming=true turnCount=5

2026-09-01T01:03:46.282Z
event=maybe_resume_success conversationId=<REDACTED>
latestTurnStatus=interrupted markedStreaming=true turnCount=5

The final assistant content and later rollout events were still present in the canonical JSONL. No canonical rollout data or history database was modified during diagnosis.

Failure mode B: completed turn is still marked as streaming without ordinal corruption

Two newer continuation threads were checked separately:

Thread C (ID redacted, rollout size about 78.9 MB)
4,168 top-level ordinal records checked
non-increasing or duplicate ordinals found: 0

Thread D (ID redacted, rollout size about 18.2 MB)
1,934 top-level ordinal records checked
non-increasing or duplicate ordinals found: 0

Despite the valid ordinal sequence, the desktop app logged completed turns as still streaming:

2026-09-02T01:00:38.080Z
event=maybe_resume_success conversationId=<REDACTED>
latestTurnStatus=completed markedStreaming=true turnCount=5

2026-09-02T01:03:40.731Z
event=maybe_resume_success conversationId=<REDACTED>
latestTurnStatus=completed markedStreaming=true turnCount=5

2026-09-02T07:28:41.117Z
event=maybe_resume_success conversationId=<REDACTED>
latestTurnStatus=completed markedStreaming=true turnCount=5

The same thread could therefore have an intact completed turn in its raw transcript while the reopened UI showed an older partial state. This suggests a stale resume/rehydration state that is independent of duplicate ordinal corruption.

Observed sequence

This is not yet a deterministic minimal reproduction, but the observed sequence was:

Use several local Codex threads, including long-running tasks.
Allow assistant turns to finish and confirm that their final responses are visible.
Close the desktop app, leave it closed overnight, or reopen/switch between tasks later.
Open the same tasks.
Some tasks display an earlier partial response instead of the newest completed response.
In one occurrence, switching tasks or using the voice UI appeared to trigger another rehydration and visual rollback. The voice action is only a possible trigger; causality is not established.

Expected behavior

A persisted assistant final response followed by normal completion should always rehydrate as completed.
A completed turn should not be marked as streaming during resume.
Top-level rollout ordinals should be unique and monotonically increasing, including after interrupted or unfinished turns.
One malformed event should not make all later valid conversation history inaccessible in the UI.
If the canonical transcript and derived history/index disagree, the app should rebuild or reconcile the projection safely, or show a recoverable diagnostic instead of silently displaying stale history.

Actual impact

The user cannot reliably tell whether long-running work has finished or has been lost.
Continuing from an older UI state risks duplicating expensive work and sending prompts against stale context.
Multiple unrelated tasks can appear affected at the same time.
The practical workaround is to preserve the old thread as read-only evidence and create a new continuation thread, which fragments project history.

Recovery attempts and checks

Confirmed that later assistant responses and events still exist in the raw session transcripts.
Restarting/reopening the app did not reliably restore the newest rendered state.
Checked continuation threads for ordinal monotonicity; the second failure mode occurs without duplicates.
Did not edit the canonical rollout JSONL, thread-history database, lock files, or system configuration during diagnosis.
Did not upgrade the app during the investigation.

Possibly related public issues

Questions for the Codex team

Is there a desktop-app fix available after version 26.825.6671.0 for stale thread rehydration or completed turns being marked as streaming?
Is there a supported way to rebuild the history projection safely from an intact paginated rollout?
Can the resume logic reconcile latestTurnStatus=completed with markedStreaming=true instead of allowing the older streaming state to win?
Would additional sanitized event windows around the duplicate ordinals or the maybe_resume_success events help?

Privacy note

This report intentionally excludes names, project titles, prompt/response content, filesystem paths, credentials, full conversation IDs, and full raw logs. Additional evidence can be provided after a separate privacy review.

I reproduced this failure on Linux with two independent affected threads and prepared a minimal tested producer fix:

offx366@55e916e298

Observed failure boundary in both cases:

token_count(N, rate_limits.primary.used_percent = float)
thread_settings_applied(N)
task_started(N + 1)

Root cause: resume-tail discovery uses ReverseJsonlScanner::scan_next::<RolloutLine>(). With serde_json/arbitrary_precision, direct deserialization can reject the flattened token_count float record, so the scanner falls back to the preceding ordinal and the resumed writer reuses N. Thread-history projection then freezes at the duplicate.

The patch scans into serde_json::Value and calls the existing decode_rollout_line, matching the projector decoder. The regression test appends a float-bearing token_count@5, resumes, and asserts the next record is ordinal 6.

Verification:

unpatched regression: fails with [0, 4, 5, 5]
patched regression: passes with [0, 4, 5, 6]
just test -p codex-rollout: 126/126 passed
just fix -p codex-rollout: passed
cargo fmt --all -- --check: passed

I attempted to open an upstream PR from offx366:fix/rollout-ordinal-decoder, but this repository restricts CreatePullRequest for external accounts. Maintainers can cherry-pick 55e916e298 or enable/invite a PR. This patch prevents the decoder-induced producer defect; self-healing of already damaged projections remains separate.

Additional macOS confirmation from read-only diagnostics on 2026-09-07 UTC. This report contains structural metadata and aggregate counts only; no private task IDs, local paths, prompts, or conversation content are included.

Environment:

Desktop: 26.901.51231 (build 8109).
Running bundled app-server: codex-cli 0.153.4, verified from the executable in the app bundle.
The production appcast still offered the installed Desktop build when checked at 06:47 UTC, so no newer production Desktop update was available in that check.

The failure persists across app-server restarts. Retained telemetry contains four affected threads and 2,161 instances of the projection warning. One unarchived task with recent activity accounts for 1,062 warnings across ten app-server processes, from September 5 05:09:38 UTC through September 7 06:38:21 UTC.

For the active case, the projection checkpoint is byte 415234111, expected ordinal 41431. The durable rollout has continued to byte 606095396, ending at ordinal 64618 and a task_complete record at September 7 06:38:21.391 UTC. The exact boundary is:

ordinal 41430: event_msg / token_count             (before resume)
ordinal 41430: event_msg / thread_settings_applied (after resume; duplicate)
ordinal 41431: event_msg / task_started

A streaming validation of the entire 190,861,285-byte suffix after the frozen checkpoint found:

23,190 complete, valid JSON records; zero JSON parse errors or blank lines.
Exactly two duplicate/regressed ordinal records, both thread_settings_applied (ordinals 41430 and 41483); no forward ordinal gaps.
159 task_started and 157 task_complete records after the frozen checkpoint.

This establishes a valid-JSON but discontinuous ordinal stream and a stale derived index while later history remains durable. It does not establish the exact writer race or parser behavior that originally caused the ordinal reuse.

There is already an upstream materialization fix, but it has not reached the stable runtime examined here:

rust-v0.153.4, thread_history_materialization.rs:193–198 returns an error for ordinal < next_ordinal.
Merged PR #42369, commit 095ac4f131e759b204fa6368dc42d2feff6eb21a, adds recovery past duplicate/regressed records and anomaly telemetry.
rust-v0.154.0-alpha.3, thread_history_materialization.rs:206–219 contains that duplicate-skipping behavior; the alpha source was inspected, not installed.

Could the existing fix be backported to the stable runtime and shipped in the production Desktop app? Please also retain regression coverage for a resumed thread with a trailing token_count(N) followed by thread_settings_applied(N), including two such discontinuities in one durable suffix.

The intended supported recovery is for a production build containing the fix to catch up from the existing projection checkpoint through normal materialization, then verify that the checkpoint reaches the complete durable suffix and recent history becomes available. This local recovery has not been executed or claimed verified: the affected rollouts, databases, and app runtime were left unchanged. No alpha build or manual database/rollout repair was used.

I can reproduce this exact issue on Windows with Codex Desktop package 26.901.6511.0_x64 (runtime 0.150.1).

Observed evidence:

The rollout remained complete and was approximately 125,006,862 bytes.
The durable history projector stopped at byte offset 12,996,041.
Logs repeatedly reported: thread history projection ... expected ordinal 289, got 288.
The duplicate record was event_msg/thread_settings_applied; the next valid record used ordinal 289.
Before recovery, the UI projection exposed only 3 turns / 93 items.
After backing up the state and advancing only the projection byte cursor past that duplicate non-message record, the same thread exposed 38 turns / 1,408 items again.
No transcript content was rewritten or removed.

User impact:
The conversation appeared to lose both recent and older work after restarting the app, and substantial additional Codex usage was consumed diagnosing and recovering from this product defect.

I attempted the in-app feedback/upload route twice with a sanitized diagnostic report and no transcript attachment, but both attempts timed out while connecting to the Sentry ingestion endpoint, so I am adding the reproduction here.

Compensation request:
Please grant two Codex usage-reset credits for the usage consumed by this incident, after account verification. Because account/session identifiers should not be posted publicly, please advise the official secure support channel where I can provide those details privately.

No thesis/PPT content, raw conversation text, account identifier, thread identifier, or local filesystem paths are included in this comment.

Consolidated status for this thread (the mechanism, the fix state on main, and the one question several of you have asked: is the fix in a stable release yet?).

Mechanism (consolidated from the 15+ reports here and the satellite issues)

The producer defect is a resumed writer reusing the final durable ordinal after an interruption. On builds before #42378, resume-tail discovery ran the backward scan through ReverseJsonlScanner::scan_next::<RolloutLine>(); with serde_json's arbitrary-precision handling, direct deserialization bails on the trailing float-bearing token_count records (decimal rate_limits.used_percent), so the scanner falls back k records and derives a stale next ordinal (N−k, k = trailing decimal-bearing records). The first post-resume append — usually thread_settings_applied (written via the raw settings-restore path, i.e. the natural first writer after a fresh-process resume), sometimes task_started — then reuses N. The projector parks exactly there: expected ordinal N+1, got N.

This matches every boundary in this thread: token_count(N) → thread_settings_applied(N) dup → task_started(N+1), with the one-trailing and two-trailing token_count variants (unravel11's cases B/E, prodan1234's second discontinuity, #43142's k=3 evidence) and the task_started(N) duplicate variant (YDLuo-1, woshishinidie). It was independently root-caused and patched by offx366's 55e916e298 (scan into serde_json::Value, decode via decode_rollout_line) — the same shape that landed upstream as #42378.

Fix state on `main` (verified against current tags)

Route rollout reads through the canonical JSON decoder #42378 (69cebb5d15, merged Sep 2): rollout reads routed through the canonical decoder → a resumed writer no longer mis-derives the ordinal → no new duplicate/regressed ordinals.
Keep SQLite history projection moving past invalid records #42369 (095ac4f131, merged Sep 2): the batch materializer now skips ordinal < next_ordinal lines with a recorded DuplicateOrRegressedOrdinal anomaly and advances the cursor (thread_history_materialization.rs:206-219 on main) → an already-stalled projection catches up on the next open/resume once running a build with this fix. prodan1234's source comparison of rust-v0.153.4 vs rust-v0.154.0-alpha.3 is exactly right.

Is it in a stable release? Not yet. Tag ancestry as of 2026-09-07: stable line ends at rust-v0.153.4 (published Sep 4) and contains neither fix (verified via git rev-list rust-v0.153.4); the newest build containing both is rust-v0.154.0-alpha.3. There is no rust-v0.154.0 stable and no 0.153.x patch with the backport yet — so for users on the stable desktop bundle, the two options are the alpha line (where #43262 empirically showed a stalled 608MB thread fully recover: 27,857 items / 42 turns reprojected) or waiting for a stable release/backport decision.

What still needs maintainers (the remaining open work)

Supported in-place repair for already-stalled projections. migrate-rollouts --apply returns already_paginated when history_mode == Paginated without checking projection consistency (rollout_migration.rs:513-517), so there is no non-destructive recovery command; the strict apply_projection path still hard-errors and bricks thread/fork and thread/revert on a stale cursor (ArthurOstapenko's fork -32603, cappallo's segment failures, YDLuo-1's frozen fork). 4Liberty's reference patch (Aug 30, tested against 63d213884d) remains the fullest published design for this: strict rebuild that consumes a duplicated no-op settings record, atomic replacement of only the affected thread's derived rows, and typed projection outcomes (repaired/unrecoverable) surfaced to the client.
A stable backport or release decision carrying Keep SQLite history projection moving past invalid records #42369+Route rollout reads through the canonical JSON decoder #42378 (per prodan1234's ask).
Regression coverage for one and two trailing token_count records plus internal-history segments (unravel11/prodan1234), and the resumed-writer test shape YDLuo-1 suggested (persist token_count@N; interrupt; resume; assert next ordinal == N+1) — upstream Route rollout reads through the canonical JSON decoder #42378's regression test covers k=1 only.

Family map for triage

This thread is the canonical issue. Same mechanism, same pre-fix runtimes: #43124 (macOS), #43182 (macOS, 0.153.4), #43142 (Windows), #43262 (Windows recovery proof), #43451 (macOS, filed today), and earlier #42999 / #42157 / #42241 / #41986; adjacent writer-serialization issues #40109 / #41079 / #41353; stale-rehydration-without-ordinal-corruption #39885. A single fix release + a repair path would close the whole cluster.

No transcript content or identifiers from any affected thread are referenced above; all claims are from the sanitized structural reports in this thread and public source/tag inspection.

Current Windows reproduction on the latest signed Desktop/runtime available on this machine (sanitized; no transcript content, credentials, local paths, or thread identifier):

Codex Desktop package: 26.901.6511.0
Bundled Codex core/CLI: 0.153.4
Windows x64
Long-running paginated Desktop thread

The latest active conversation hit the same producer/projection failure during a normal later turn start after an earlier completed segment. Read-only rollout metadata at the failure boundary is:

ordinal 1073  response_item/custom_tool_call_output
ordinal 1074  event_msg/token_count
ordinal 1074  event_msg/task_started      <-- duplicate
ordinal 1075  turn_context

The durable history projector then repeatedly logged:

thread history projection ... expected ordinal 1075, got 1074

Read-only projection state confirms the cursor is parked exactly at the duplicate task_started record:

next_rollout_ordinal     = 1075
next_rollout_byte_offset = 95782442

That byte offset is exactly the start of the second ordinal-1074 record. The canonical rollout remains valid JSON and continues beyond the duplicate. thread_history_1.sqlite returns PRAGMA quick_check = ok, so this is again a logical writer/projection inconsistency rather than structural SQLite corruption.

This reproduction is useful because the duplicate record is task_started, not thread_settings_applied, and it occurs on shipped 0.153.4, confirming the writer-side ordinal reuse remains possible in that signed runtime. No repair/database mutation has been attempted in this diagnostic pass.

I can corroborate this exact failure signature on Linux with codex-cli 0.153.4.
It occurred in two independent long-running local desktop tasks, including a replacement
task created after the first task became unusable through Remote.

For the most recent occurrence:

byte 14273303: ordinal 3372, event_msg/token_count
byte 14274120: ordinal 3372, event_msg/thread_settings_applied  <-- duplicate
byte 14276031: ordinal 3373, event_msg/task_started

thread_history_projection_state before recovery:
  next_rollout_byte_offset = 14274120
  next_rollout_ordinal     = 3373

max projected item ordinal before recovery = 3370
rollout at diagnosis                         = 7,031 valid JSON records
rollout ordinal range                        = 0..7029

The duplicate above was the rollout's only non-monotonic event. The task-history API
returned the older turn as inProgress, and Remote repeatedly showed an old commentary
as the latest update even though the rollout continued normally.

I also tested recovery on this affected task. After making a transactionally consistent
backup of thread_history_1.sqlite, I atomically advanced only this task's projection
cursor from the duplicate non-message record to the immediately following expected
ordinal-3373 task_started record:

before: (byte_offset=14274120, next_ordinal=3373)
after:  (byte_offset=14276031, next_ordinal=3373)

I did not rewrite, delete, or renumber any rollout record. The projector immediately
caught up to the live end without an app restart. A later verification showed:

projection byte offset = 30051958
rollout byte length     = 30051958
next expected ordinal  = 7144
rollout last ordinal    = 7143

This is not a recommendation that users edit SQLite manually; it is additional
localization evidence that the canonical rollout is intact and that the permanent
failure is caused by the projector being unable to reconcile the duplicated non-message
ordinal. A supported repair path could safely recognize this exact metadata pattern,
preserve all message records, and resume at the next expected ordinal.

Environment:

codex-cli 0.153.4
Linux 7.0.0-31-generic x86_64
Codex desktop local task with Remote Control

I can provide a further-redacted event window and projection metadata if useful. I am
not attaching the raw rollout or database because they contain private conversation and
workspace data.

Additional corroboration from a local macOS desktop task (sanitized):

ChatGPT/Codex desktop, bundled Codex runtime 0.153.4
Local project; other sessions in the same project can start Side Chat normally
One long-running existing task fails Side Chat consistently. The composer reaches thread/fork and receives error -32603:

failed to prepare paginated fork: thread-store internal error:
thread history projection for <redacted> expected ordinal 4940, got 4939

The failure reproduces across multiple Side Chat attempts. Configuration loaded normally; no raw transcript, task identifier, project path, credentials, or logs are included. This is additional evidence that an ordinal/projection inconsistency blocks the fork-based Side Chat flow for already-affected local tasks.

Additional confirmed occurrence on Linux with Codex CLI/app-server 0.153.4. User-confirmed trigger: the host ran out of memory, and the user rebooted it while the affected session was still running. The session was subsequently resumed. This appeared to the user as a remote connection failing to update an existing session. The duplicate record in this case is task_started.

Environment and impact

Linux 7.0.0-31-generic, x86_64.
App server managed by a systemd user service, launched with codex app-server --remote-control --listen unix://.
Existing thread uses history_mode = paginated.
The service was running and received an iOS remote client's thread/resume request. The durable rollout contained newer completed turns, but the history projection remained at an older turn.

Exact failure boundary

Sanitized top-level event metadata, with zero-based physical record positions:

Record position	UTC timestamp (2026-09-08)	Ordinal	Event
2473	02:45:53.461	2473	response_item/function_call_output
2474	02:45:57.212	2474	event_msg/token_count
2475	03:15:37.371	2474	event_msg/task_started
2476	03:15:39.223	2475	response_item/message (user)

The systemd service started after the host reboot at 03:07:37 UTC, between the two ordinal-2474 records. Every subsequent ordinal is one behind its physical record position.

Repeated log from codex_thread_store::local::live_writer:

failed to project durable rollout for <thread-id>:
thread-store internal error:
thread history projection for <thread-id> expected ordinal 2475, got 2474

Projection state at diagnosis:

next_rollout_ordinal     = 2475
next_rollout_byte_offset = 12825091
rollout record count    = 3146
rollout byte length     = 15329346

There are 671 records after the projection cutoff, including completed turns through 03:53:47 UTC. The last indexed turn remained inProgress in the live projection database.

Controlled recovery test

We preserved the original rollout and tested separate, isolated copies of the state/history databases with the same 0.153.4 binary:

Original copy: thread/resume, followed by thread/turns/list, still returned the older history. Projection remained at ordinal 2475 / byte 12825091.
Corrected copy: Increased only each top-level ordinal from physical position 2475 onward by one. Verified all JSON payloads, timestamps, record count, and byte lengths were unchanged. After thread/resume, the projection advanced to ordinal 3146 / byte 15329346, and thread/turns/list returned the latest three completed turns.

No projection rows were manually removed or edited in this test, and no model turn was started. The live rollout has not been modified.

This isolates the stale remote history to the ordinal/projection failure. The out-of-memory condition and reboot during the running session are user-reported; the reboot/resume event boundary is corroborated by local timestamps. We have not established whether the kernel OOM killer terminated a Codex process before reboot. This is not a deterministic minimal reproduction or proof of the precise writer-side cause. A supported recovery path and release guidance for affected 0.153.4 sessions would be helpful.

This report includes structural metadata only; private conversation contents, credentials, local paths, and thread identifiers are omitted.

Additional confirmed macOS instance, with a controlled prerelease comparison and a completed, user-verified recovery on September 8, 2026.

This report was prepared with Codex and posted with the affected user's permission. It contains structural evidence only; no conversation text, task identifiers, project names, local paths, credentials, databases, or raw logs are attached.

Environment and failure

macOS 26.5.1, Apple Silicon.
Signed Desktop bundle: 26.901.51231, build 8109, bundle identifier com.openai.codex.
Bundled CLI/app-server: 0.153.4, rechecked after recovery.
Four local threads using paginated history had stopped displaying later messages.
Each affected canonical JSONL was parseable and contained a single duplicated top-level ordinal on event_msg/thread_settings_applied. The projection checkpoint stopped exactly at that discontinuity, with codex_thread_store::local::live_writer reporting expected ordinal N, got N-1. Later user and assistant messages were still in the canonical transcripts.

Controlled copy tests

Testing used fresh copies of the state/history databases and complete-record transcript snapshots in network-disabled Docker containers, with the official @openai/codex packages. No credentials were copied and no successful model request was made.

Test	Observed result
`0.153.4`, affected unmodified snapshots	Reproduces the stalled projection
Remove only the derived index in a disposable copy, then resume on `0.153.4`	Does not repair it; projection fails again at the duplicate
Built-in migration on an already-paginated copy	Reports `already_paginated`
`0.154.0-alpha.6`, unmodified transcript copies, resume only	Recovers existing history without transcript edits, settings saves, or new turns
Shut down the recovery backend; paginate recovered copies with `0.153.4`	All original visible message IDs and text match
Separate persistence tests using `0.154.0-alpha.6`, including restart and another synthetic message	Passed for the three snapshots tested

Recurrence on stable was demonstrated separately: after prerelease-assisted recovery of three copied threads, submitting one synthetic message per copy using 0.153.4 caused one thread to stall again. Its captured pre-resume tail was a token-count event. The projector logged ten instances of expected ordinal 38553, got 38552; the new message was not projected. The other two copies persisted their messages. The driver exit code alone did not detect this failure; database inspection did.

These are controlled tests using affected snapshots, not a minimal reproduction from an empty installation or proof that all recurrence mechanisms are fixed.

Completed recovery and native Desktop confirmation

With Desktop and all other live-history holders closed, fresh backups were taken. The prerelease rebuilt only disposable copies; after verification, the four allowlisted threads' derived rows were imported in one guarded SQLite transaction. No canonical transcript, state database, application binary, or unrelated task was replaced or edited by the recovery.

All four final checks passed:

Anonymous thread	Original visible messages verified	Pages read by `0.153.4`
A	1,064	138
B	1,752	154
C	251	28
D	1,364	130

The final live report recorded applied: true and verified: true: 4,431 original visible messages matched by ID and text, all four checkpoints reached the captured transcript ends, transcript sizes/SHA-256 hashes were unchanged, unrelated rows and migration records matched the fresh backup, and SQLite integrity checks passed. After reopening Desktop, the user confirmed that the previously missing messages were visible.

This is recovery evidence, not a recommendation to edit databases or a permanent fix for 0.153.4.

Release guidance requested

The public fixes #42369 and #42378 were merged on September 2. Both merge commits are ancestors of rust-v0.154.0-alpha.6 and are not ancestors of rust-v0.153.4. The stable tag's materializer still errors on a regressed ordinal, and its ordinal reader still uses the direct RolloutLine scanner. The latest stable GitHub release checked for this report remains 0.153.4.

Please clarify the first supported Desktop/stable build shipping both changes, whether already-stalled histories recover automatically after upgrading, and whether a supported per-thread integrity/recovery operation will be available. The successful prerelease tests do not establish the exact initiating cause for these four original failures or certify all prerelease behavior.

Deterministic Windows reproduction: `0.0` versus `0`, without a crash or model call

I reproduced the producer defect through the public app-server API with a five-record synthetic paginated rollout. This supplies a small deterministic trigger for this issue: a forced interruption, concurrent writer, long conversation, or model request is not needed for this reproduction.

Tested on Windows 11 x64, build 26200, on September 9, 2026:

Installed Desktop package: OpenAI.Codex 26.901.6511.0.
Actual running backend executable: codex-cli 0.153.4, checked with --version. The older CLI version recorded when a thread was created is not used as the current runtime version.
Comparison binary: official npm @openai/codex@0.154.0-alpha.8-win32-x64; package SHA-512 integrity checked and executable version verified. This was installed only in a separate test directory.

Producer reproduction

Create a fresh isolated CODEX_HOME, an offline local model-provider configuration, and a newline-terminated JSONL fixture containing these records:

0  session_meta: history_mode = paginated
1  event_msg/task_started
2  event_msg/item_completed: synthetic user message
3  event_msg/task_complete
4  event_msg/token_count: rate_limits.primary.used_percent = 0.0

Then call initialize with experimentalApi: true, initialized, thread/resume with the fixture path and excludeTurns: true, and thread/settings/update with effort: "high". Close the app-server normally through stdin EOF. No turn/start request is sent.

Binary	Tail JSON number	Resulting ordinals	Projection
`0.153.4`	`0.0`	`0,1,2,3,4,4`	Stalls: `expected ordinal 5, got 4`
`0.153.4`	`0`	`0,1,2,3,4,5`	Reaches EOF
`0.154.0-alpha.8`	`0.0`	`0,1,2,3,4,5,6`	Reaches EOF; both appended settings records have unique ordinals

The last record is serialized as a decimal in the first and third cases, and as an integer in the control. Each run uses a separate home. The stable binary itself writes the duplicate thread_settings_applied; the fixture does not contain a duplicate. The preceding synthetic turn is explicitly completed.

All three public RPCs (thread/resume, thread/settings/update, thread/turns/list) report success in the failing stable case. Detecting the failure requires inspecting the persisted ordinal sequence/checkpoint or the warning, not just the RPC status.

Recovery comparison

In a separate test, I retained the duplicate produced by 0.153.4, closed the server, and appended three synthetic records with ordinals 5,6,7 representing a second completed turn. Two disposable homes received byte-identical rollout copies and equivalent copies of the frozen projection state; only the state database's local rollout path was adjusted for each home.

Test	Visible turns/items	Checkpoint
Resume with `0.153.4`	1 turn / 1 item	Still byte 1546, expected ordinal 5; rollout is 3174 bytes
Resume with `0.154.0-alpha.8`	2 turns / 2 items	Reaches EOF: byte 4119, expected ordinal 9
Close alpha and resume the recovered copy with `0.153.4`, without updating settings or starting a turn	Both original synthetic messages remain readable	Still at EOF

Alpha appended a settings record at ordinal 8. The entire pre-existing 3174-byte canonical prefix remained byte-identical, verified by SHA-256; no duplicate or message record was edited, renumbered, or deleted. SQLite quick_check returned ok. This last stable read verifies persistence only; it does not establish that further writes on stable cannot reproduce the defect.

Root cause and existing fixes

The decimal/integer control supports the decoder mechanism already discussed here and addressed by #42378. In rust-v0.153.4 ordinal discovery, the reverse scanner directly deserializes RolloutLine and ignores rejected records. Rejection of the float-bearing final record lets it derive the next ordinal from the earlier record. The subsequent settings append therefore reuses the final durable ordinal. This is consistent with the flattened-envelope/nested-decimal decoding problem described and fixed by #42378.

The second failure is explicit in the stable materializer: a regressed ordinal returns an error and prevents later valid history from being projected. #42369 adds recovery past that boundary. Both fixes were already merged; I am reporting controlled binary verification, not proposing those patches as new work.

The affected real desktop task has the same structural boundary, token_count(38691) -> thread_settings_applied(38691) -> task_started(38692), exactly at the frozen projection byte offset. Its later conversation records remained valid JSON on disk. No real conversation content, task identifiers, local paths, credentials, or raw logs are included here.

As of this check, npm's latest tag still resolves to 0.153.4. Could the applicable prevention and recovery fixes be shipped/backported to the stable desktop runtime, and the first stable desktop build containing both be identified?

Prepared and tested with Codex, and posted with the affected account owner's authorization. The actual desktop task and its databases were not modified by these reproduction tests.

Self-contained Python reproducer (standard library; tested on the binaries above)

Save as reproduce_history_ordinal.py and run each case against a fresh disposable home:

python reproduce_history_ordinal.py PATH_TO_STABLE_CODEX decimal TEST_WORK_DIR
python reproduce_history_ordinal.py PATH_TO_STABLE_CODEX integer TEST_WORK_DIR
python reproduce_history_ordinal.py PATH_TO_ALPHA_CODEX decimal TEST_WORK_DIR

The script creates a new isolated home under the supplied work directory, uses a local non-authenticated dummy provider, disables hooks and analytics, and makes no model call. It prints the persisted ordinals and checkpoint. It does not open the normal application data directory. The recovery comparison above was a separate experiment; this script reproduces the producer/control cases.

"""Reproduce Codex resume ordinal reuse using synthetic data and no model call.

Usage: python reproduce_history_ordinal.py PATH_TO_CODEX decimal|integer WORK_DIR
Each run creates a new isolated CODEX_HOME under WORK_DIR; inspect the printed path.
Only initializes/resumes a synthetic thread, updates its settings, and reads history.
"""

import json
import os
import pathlib
import queue
import sqlite3
import subprocess
import sys
import tempfile
import threading

binary = pathlib.Path(sys.argv[1]).resolve()
variant = sys.argv[2]
assert variant in ('decimal', 'integer')
parent = pathlib.Path(sys.argv[3]).resolve()
parent.mkdir(parents=True, exist_ok=True)
home = pathlib.Path(tempfile.mkdtemp(prefix=f'ordinal-{variant}-', dir=parent))
workspace = home / 'workspace'
workspace.mkdir()
tid = '11111111-1111-4111-8111-111111111111'
stamp = '2026-09-09T00:00:00.000Z'
rollout = home / 'sessions/2026/09/09' / f'rollout-2026-09-09T00-00-00-{tid}.jsonl'
rollout.parent.mkdir(parents=True)

def record(ordinal, kind, payload):
    return {'timestamp': stamp, 'ordinal': ordinal, 'type': kind, 'payload': payload}

records = [
    record(0, 'session_meta', {'id': tid, 'session_id': tid, 'timestamp': stamp,
        'cwd': str(workspace), 'originator': 'history-repro', 'cli_version': '0.153.4',
        'source': 'vscode', 'model_provider': 'repro', 'base_instructions': {'text': 'Synthetic fixture.'},
        'dynamic_tools': [], 'history_mode': 'paginated'}),
    record(1, 'event_msg', {'type': 'task_started', 'turn_id': 'fixture-turn-1',
        'started_at': 1788912000, 'model_context_window': None, 'collaboration_mode_kind': 'default'}),
    record(2, 'event_msg', {'type': 'item_completed', 'thread_id': tid, 'turn_id': 'fixture-turn-1',
        'item': {'type': 'UserMessage', 'id': 'fixture-user-1',
                 'content': [{'type': 'text', 'text': 'Synthetic fixture message'}]},
        'started_at_ms': 1788912000000, 'completed_at_ms': 1788912000001}),
    record(3, 'event_msg', {'type': 'task_complete', 'turn_id': 'fixture-turn-1',
        'last_agent_message': None, 'started_at': 1788912000, 'completed_at': 1788912001,
        'duration_ms': 1000}),
    record(4, 'event_msg', {'type': 'token_count', 'info': None, 'rate_limits': {
        'primary': {'used_percent': 0.0 if variant == 'decimal' else 0,
                    'window_minutes': 300, 'resets_at': 1800000000}}}),
]
rollout.write_text(''.join(json.dumps(r, separators=(',', ':'))+'\n' for r in records), encoding='utf-8')
(home / 'config.toml').write_text('''model = "fixture-model"
model_provider = "repro"
[model_providers.repro]
name = "Offline reproduction"
base_url = "http://127.0.0.1:9/v1"
wire_api = "responses"
requires_openai_auth = false
[features]
hooks = false
[analytics]
enabled = false
''', encoding='utf-8')
env = {k: v for k, v in os.environ.items()
       if not any(s in k.upper() for s in ('API_KEY', 'AUTH_TOKEN', 'ACCESS_TOKEN'))}
env['CODEX_HOME'] = str(home)
env['HTTP_PROXY'] = env['HTTPS_PROXY'] = env['ALL_PROXY'] = 'http://127.0.0.1:9'
env['NO_PROXY'] = '127.0.0.1,localhost'
messages = queue.Queue()
results = {}
with (home / 'stderr.log').open('w', encoding='utf-8') as stderr:
    process = subprocess.Popen([str(binary), 'app-server', '--stdio'], cwd=workspace, env=env,
        stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=stderr, text=True, encoding='utf-8',
        creationflags=getattr(subprocess, 'CREATE_NO_WINDOW', 0))

    def reader():
        for line in process.stdout:
            try:
                messages.put(json.loads(line))
            except ValueError:
                pass

    threading.Thread(target=reader, daemon=True).start()

    def send(value):
        process.stdin.write(json.dumps(value)+'\n')
        process.stdin.flush()

    def call(request_id, method, params):
        send({'id': request_id, 'method': method, 'params': params})
        while True:
            value = messages.get(timeout=30)
            if value.get('id') == request_id:
                results[method] = value
                if 'error' in value:
                    raise RuntimeError(value['error'])
                return value['result']

    try:
        call(1, 'initialize', {'clientInfo': {'name': 'history-repro', 'version': '1.0.0'},
                              'capabilities': {'experimentalApi': True}})
        send({'method': 'initialized', 'params': {}})
        call(2, 'thread/resume', {'threadId': tid, 'path': str(rollout), 'excludeTurns': True})
        call(3, 'thread/settings/update', {'threadId': tid, 'effort': 'high'})
        call(4, 'thread/turns/list', {'threadId': tid, 'itemsView': 'full', 'limit': 100})
    finally:
        process.stdin.close()
        try:
            process.wait(timeout=15)
        except subprocess.TimeoutExpired:
            process.terminate()
            process.wait(timeout=10)

written = [json.loads(line) for line in rollout.read_text(encoding='utf-8').splitlines()]
ordinals = [line.get('ordinal') for line in written]
assert ordinals[:5] == [0, 1, 2, 3, 4]
assert len(ordinals) > 5, 'No new record was persisted; reproduction inconclusive'
with sqlite3.connect((home / 'thread_history_1.sqlite').as_uri()+'?mode=ro', uri=True) as db:
    checkpoint = db.execute('SELECT next_rollout_byte_offset,next_rollout_ordinal FROM '
                            'thread_history_projection_state WHERE thread_id=?', (tid,)).fetchone()
summary = {'binary_version': subprocess.check_output([str(binary), '--version'], text=True).strip(),
           'variant': variant, 'home': str(home), 'ordinals': ordinals,
           'record_types': [line['payload'].get('type', line['type']) for line in written],
           'checkpoint': checkpoint, 'rollout_bytes': rollout.stat().st_size,
           'duplicate_ordinal': len(set(ordinals)) != len(ordinals),
           'rpc_errors': [value['error'] for value in results.values() if 'error' in value]}
(home / 'results.json').write_text(json.dumps({'summary': summary, 'rpc': results}, indent=2), encoding='utf-8')
print(json.dumps(summary, indent=2))

Disclosure: this comment was prepared by Codex from local structural diagnostics and is being posted with the affected account owner's explicit approval. It contains no task/thread identifier, project name, prompt, conversation content, credential, private path, or raw log.

Additional current confirmation on the stable 0.153.4 runtime, plus a narrowly validated recovery result.

Environment and topology:

macOS Desktop application 26.901.51231 (build 8109)
remote Ubuntu 24.04 x86_64 App Server / CLI 0.153.4
paginated history accessed from Desktop over the remote App Server connection
observed and recovered on September 9, 2026

The frozen boundary was structurally identical to the producer defect consolidated above:

token_count(909)
thread_settings_applied(909)  # duplicate
task_started(910)

The projection cursor was parked at byte 34332222, expecting ordinal 910, exactly where the second ordinal 909 began. The canonical rollout remained valid and grew to 62557748 bytes, leaving 28225526 bytes and nine later task_complete events unprojected. thread/read exposed only the old ten-turn prefix. SQLite quick_check was ok, and codex doctor reported database integrity/inventory status without detecting this logical cursor mismatch.

A separate isolated copy of an earlier occurrence remained frozen on 0.153.4 after successful-looking thread/read and thread/resume; migrate-rollouts returned already_paginated and did not repair it.

Verified recovery, after consistent SQLite/rollout backups and confirmation that no task scope or App Server writer was active:

Advance only next_rollout_byte_offset by the duplicated record's exact 1,555-byte length.
Keep next_rollout_ordinal at 910.
Do not edit or renumber the canonical rollout.
Resume the thread without starting a turn.

The projector then consumed the remaining valid records and reached exact EOF at 62557748 / 2099. quick_check remained ok, and the API-visible history increased from 10 to 20 turns, including the previously hidden recent completed work.

This corroborates that stable 0.153.4 still has both the producer and recovery gap. It also validates the core behavior expected from #42378 and #42369: prevent the duplicate on resume, and skip/reconcile the already-persisted regressed metadata record so an existing projection can catch up without source-rollout mutation. A supported per-thread repair/doctor path is still important for users who cannot safely perform internal SQLite surgery.

Additional macOS occurrence: history still frozen after desktop update

Prepared with Codex and posted at the affected account owner's explicit request. This report contains structural diagnostics only. It excludes conversation text, company and project names, contact information, task identifiers, private filesystem paths, credentials, databases, and raw logs.

Environment

macOS 26.5, build 25F71.
Desktop before the update: 26.901.22334, build 7746.
Desktop after the update: 26.903.71938, build 8576.
Bundled executable after the update: codex-cli 0.153.4, checked directly with --version.
Observed again on September 10, 2026, after the user updated and reopened the app.

User-visible impact

An existing long-running local task repeatedly reopens at an old assistant message from September 3. Messages and work from the following days disappear from the rendered history. The raw session JSONL still contains later user messages through September 10, including the current complaint. Before the update, the task-history reader independently returned the September 3 turn as its newest turn even while task metadata was current.

The update did not recover this task: the same projection checkpoint and the same live error were reverified afterward. This is persistent apparent history loss, not evidence that the later transcript was deleted.

Exact failure boundary

Sanitized metadata from consecutive records in the affected canonical JSONL:

Physical line (one-based)	UTC timestamp	Top-level ordinal	Event
5739	2026-09-03T15:58:43.507Z	5738	`event_msg/token_count`
5740	2026-09-03T15:59:03.225Z	5738	`event_msg/thread_settings_applied`
5741	2026-09-03T15:59:03.228Z	5739	`event_msg/task_started`

These are two distinct records with the same ordinal, not a duplicated display of one record.

Read-only inspection of thread_history_1.sqlite after the update showed:

thread_history_projection_state.next_rollout_byte_offset = 107722943
thread_history_projection_state.next_rollout_ordinal = 5739
latest indexed item ordinal = 5736
latest indexed item time = 2026-09-03 08:58:43 America/Los_Angeles

At 2026-09-10 08:54:38 Pacific, the canonical rollout was 1,082,234,375 bytes and still being appended to, far beyond that frozen checkpoint.

Live warnings from codex_thread_store::local::live_writer continued after the update, including at 08:54:33 and 08:54:38 Pacific:

failed to project durable rollout for <thread-id>:
thread-store internal error:
thread history projection for <thread-id> expected ordinal 5739, got 5738

Reproduction scope and unknowns

The observed user path is to continue the affected task, reopen it or update/restart the desktop app, and see the old history again. The persisted duplicate boundary and blocked projection are confirmed. The original action that created the duplicate ordinal has not been established; this report does not claim a deterministic producer reproduction, a concurrency cause, or a conversation-size cause.

No transcript renumbering, record deletion, history-database clearing, migration, or prerelease recovery has been attempted during this diagnosis. No full session or raw log was uploaded.

Requested resolution

Please address both creation of duplicate ordinals and safe recovery of already-affected tasks. Updating the desktop app alone has not repaired this instance. A recovery path needs to preserve both distinct records and all later turns, make the history reader catch up, and keep history current after reopening. Until then, the app should make the stalled projection visible rather than presenting an old turn as current history without an actionable warning.

Local deduplication marker: history-projection-20260910-5739-5738.

Additional confirmation on Linux: two local paginated threads show the same duplicate-ordinal failure as this issue.

Environment

Desktop app: 26.901.20858.
Bundled CLI/app-server used for diagnosis: 0.153.0-alpha.5.
Arch Linux x86_64, kernel 7.1.9-arch1-2, Hyprland/Wayland. The Linux Debian package was converted to an Arch package; this is outside the officially supported distributions.
Observed September 8–10, 2026. The newer Linux desktop package 26.903.71938 has not been tested.

Observed failure

After a forced reboot during ongoing work, both conversation histories remained at their earlier cutoff even though their canonical JSONL files kept receiving later turns.

The failure boundaries match the pattern reported here:

Thread A:
token_count             ordinal 4695
thread_settings_applied ordinal 4695  <- repeated
task_started            ordinal 4696

Thread B:
token_count             ordinal 4030
thread_settings_applied ordinal 4030  <- repeated
task_started            ordinal 4031

The app logs repeatedly report this error for Thread A (identifier redacted):

failed to project durable rollout for THREAD_A:
thread-store internal error:
thread history projection for THREAD_A expected ordinal 4696, got 4695

Captured state	Thread A	Thread B
Original rollout bytes	221,008,900	62,898,547
Valid JSONL records	10,438	6,425
Projection byte checkpoint	50,598,514	28,870,801
Expected next ordinal	4696	4031
Turns returned by native history API	20	23

All original JSONL records parsed successfully. quick_check passed for both the state and history databases. The checkpoint is immediately before the repeated ordinal. The association with the reboot is observed; I have not constructed a deterministic reproduction of the initiating writer failure.

In both captured boundaries, the preceding token_count has rate_limits.primary.used_percent serialized as a decimal with a .0 suffix. This is consistent with the decoder mechanism described in #41566 (comment); I have not independently run that decimal/integer control on this binary.

Controlled recovery evidence

Copies were tested with the same bundled app-server inside a network-isolated bubblewrap environment, with the real filesystem read-only. No model turn was started and no live conversation data was modified.

Reopening the existing copied history returned the same stale 20/23 turns.
Removing only the two threads' derived history records in the copy and resuming did not automatically reconstruct them: the history API returned zero turns.
The migration CLI classified the original histories as already_paginated.
On disposable copies only, reclassifying the histories as legacy and using the native migrator to regenerate ordinals and the projection produced 33 turns for each thread.
After another isolated app-server restart, thread/turns/list still returned 33/33 turns. The reconstructed database passed integrity_check. A full event comparison found no changes to user/assistant message content; differences were ordinal renumbering and absent-to-null serialization normalization.

This is recovery evidence from disposable copies, not a proposed general repair procedure.

Could maintainers confirm which desktop/bundled-runtime release includes the recovery handling in #42369, and whether upgrading automatically recovers already-stalled paginated checkpoints? A supported per-thread verification/recovery command would also help.

No raw transcripts, database files, project names, filesystem paths, credentials, or private session identifiers are attached.

Two Windows recovery failures after unclean shutdowns on 0.153.4

Two local paginated chats show stale history while their JSONL transcripts continue receiving new records. One matches this issue's duplicate-ordinal pattern; the other has a zero-filled region and a stale byte checkpoint. They followed the same series of unclean shutdowns, but a shared code-level cause has not been established.

Environment

Windows 11 Pro 24H2, x64, build 10.0.26100.9168.
Desktop release 26.903.71938; installed MSIX package 26.903.9818.0.
Bundled executable: codex-cli 0.153.4.
Both chats use history_mode = paginated.

Case A: duplicate ordinal blocks projection and forking

Consecutive records at the failure boundary:

Physical line	Ordinal	Event	Byte start
6254	6253	token_count	183892431
6255	6253	thread_settings_applied	183893335
6256	6254	task_started	183895356

The projection checkpoint is exactly at the second ordinal 6253:

next_rollout_byte_offset = 183893335
next_rollout_ordinal = 6254

Every captured line parses as JSON, and this is the only ordinal discontinuity. The preceding token_count contains decimal serialization of used_percent as 0.0, consistent with the decoder discussion here; that mechanism has not been independently tested on this binary.

The desktop repeatedly fails thread/fork with code -32603:

failed to prepare paginated fork: thread-store internal error:
thread history projection for <thread A> expected ordinal 6254, got 6253

The live writer reports the same mismatch while later messages continue being appended.

Case B: zero-filled region and a checkpoint inside a valid record

Physical line	Ordinal	Record	Byte range [start, end)
14943	14942	custom_tool_call	[113061345, 113062726)
14944	none	159,609 NUL bytes followed by one LF	[113062726, 113222336)
14945	14943	thread_settings_applied	[113222336, 113224369)
14946	14944	task_started	[113224369, 113224618)
14950	14948	message	[113234671, 113235099)
14954	14952	item_completed	[113237882, 113396594)
14955	14953	item_completed	[113396594, 113466951)

The invalid physical line is exactly b"\x00" * 159609 + b"\n". Excluding that line, valid records have a continuous ordinal sequence. This case has no duplicate ordinal.

The saved checkpoint is:

next_rollout_byte_offset = 113335182
next_rollout_ordinal = 14948

That byte offset is 97,300 bytes into the valid ordinal-14952 record. The actual current ordinal-14948 record starts at byte 113234671.

The materializer rejects the partial record at its stale checkpoint:

line_start_byte_offset=113335182
line_end_byte_offset=113396594
expected_ordinal=14948
error=expected value at line 1 column 1

It then reports:

thread history projection for <thread B> expected ordinal 14948, got 14953;
1 rejected rollout lines cannot cover that gap

Two completed command outputs remain in SQLite at indexed ordinals 14944 and 14945, but their original item_completed events are absent from the current JSONL. Those outputs contain 57,094 and 22,513 characters. The current transcript now assigns those ordinals to post-restart records. A recovery that simply discards the projection could lose that retained content.

Shutdown correlation

The Windows System log confirms three unclean shutdowns on 2026-09-10. Times below are UTC.

Previous shutdown reported by Event 6008	Subsequent OS start recorded by Event 12
09:59:57	10:02:48.889
10:02:59	10:04:47.926
11:04:59	11:08:50.842

Each boot also has Kernel-Power Event 41 with BugcheckCode=0. Case A's damaged transition spans the first outage episode and resume at 10:23:17. Case B's invalid region precedes resume at 11:11:13 after the later reboot. These establish correlation with unclean shutdowns, not the exact power-loss instant or a proven writer root cause.

Validation and recovery status

SQLite inspection used read-only connections. The shared history database passes PRAGMA quick_check.
Later messages remain readable in both raw transcripts.
migrate-rollouts without --apply reports already_paginated and bytes_processed: 0 for Case A.
No source transcript or history database has been repaired or reset.
No deterministic producer reproduction has been attempted through another forced shutdown.

Expected behavior: prevent ordinal reuse after interruption, detect stale or invalid checkpoints, and provide recovery for already-affected paginated chats. Recovery should preserve any indexed content missing from the surviving transcript. A stalled projection should produce an actionable warning rather than silently showing old history as current.

Increment the duplicate record and every subsequent top-level ordinal by one.

Validate the repaired rollout as a continuous sequence (0..5280, 5,281 ordinal-bearing records, zero anomalies).

Remove only this thread's generated history-projection rows and let the app server rebuild them.

this works for ChatGPT Version 26.908.40834 too

Independent Windows reproduction of the same failure pattern.

Environment

Codex Desktop package: OpenAI.Codex_26.908.9136.0_x64
Codex CLI: 0.150.1
OS: Microsoft Windows NT 10.0.19045.0, x64
Affected thread ID: 01a08562-f5f4-7de1-8e38-418a342d5c43
Stuck turn ID: 01a08b55-caf2-7a11-ad52-a342e9290f7b

User-visible symptom

After every desktop app restart, the thread returns to the same old conversation boundary from September 10. The visible last turn has no duration and appears interrupted, while later work from September 12–15 is missing from the UI. The project files produced by those later turns remain present.

Verified lifecycle state

Read-only SQLite inspection shows the stuck turn as:

status       = inProgress
started_at   = 2026-09-10 20:40:50 Asia/Shanghai
completed_at = NULL
duration_ms  = NULL

The outer thread is currently reported as idle, while read_thread exposes the old turn as interrupted. Newest-first history nevertheless stops at that old turn.

Exact duplicate-ordinal boundary

The canonical rollout contains:

Physical line	Ordinal	Event	UTC timestamp
2281	2280	`event_msg/token_count`	2026-09-10T12:47:06Z
2282	2280	`event_msg/thread_settings_applied`	2026-09-12T08:29:08Z
2283	2281	`event_msg/task_started`	2026-09-12T08:29:08Z

There is no task_complete between the earlier task_started and this later task start.

The history projection cursor stops at exactly this boundary:

next_rollout_ordinal     = 2281
next_rollout_byte_offset = 15345491

Persisted tail

The original rollout remains valid and continues to:

physical lines = 4503
size           = 32,312,359 bytes
latest event   = event_msg/task_complete
latest time    = 2026-09-15T07:52:10Z

The unprojected tail contains 10 later task starts, 10 matching task completions, 10 user messages, and 46 visible assistant messages. This independently confirms that later conversation data was durably written but hidden by the frozen projection.

A byte-identical SHA-256 backup was created before inspection. Neither the rollout nor either SQLite database was modified, and no lock was removed. Full conversation text, local paths, source code, credentials, and raw databases are intentionally omitted for privacy.

This looks like another direct reproduction of the unfinished-turn → duplicate thread_settings_applied ordinal → permanently frozen history projection sequence described in the issue. A supported non-destructive reindex/repair path would be very helpful in addition to preventing new duplicate ordinals.

Codex paginated rollout can emit duplicate ordinal after unfinished turn, permanently freezing thread history projection #41566

Description

What version of the Codex App are you using (From “About Codex” dialog)?

What subscription do you have?

What platform is your computer?

What issue are you seeing?

What steps can reproduce the bug?

What is the expected behavior?

Additional information

Activity

github-actions commented on Aug 29, 2026

SuperReaper1999 commented on Aug 29, 2026

mikezio commented on Aug 29, 2026

4Liberty commented on Aug 30, 2026

Codex paginated-history producer and self-healing projection defect

Sanitized evidence

Tested reference patch

Regression results

App-server and renderer integration design

Release acceptance

alexandrusavin commented on Aug 31, 2026

Jiaming-Zhu commented on Aug 31, 2026

YDLuo-1 commented on Sep 1, 2026

cappallo commented on Sep 1, 2026

hajimeno-ipoo commented on Sep 1, 2026

xhplyf618 commented on Sep 2, 2026

Summary

Environment

Failure mode A: duplicate ordinal followed by stale/frozen history

Failure mode B: completed turn is still marked as streaming without ordinal corruption

Observed sequence

Expected behavior

Actual impact

Recovery attempts and checks

Possibly related public issues

Questions for the Codex team

Privacy note

offx366 commented on Sep 2, 2026

7 remaining items

prodan1234 commented on Sep 7, 2026

rocoking331-netizen commented on Sep 7, 2026

argszero commented on Sep 7, 2026

Mechanism (consolidated from the 15+ reports here and the satellite issues)

Fix state on main (verified against current tags)

What still needs maintainers (the remaining open work)

Family map for triage

PracticalSwan commented on Sep 7, 2026

Gonzo15918 commented on Sep 8, 2026

adolphinvx commented on Sep 8, 2026

czakian commented on Sep 8, 2026

Environment and impact

Exact failure boundary

Controlled recovery test

Martin9204 commented on Sep 8, 2026

Environment and failure

Controlled copy tests

Completed recovery and native Desktop confirmation

Release guidance requested

cookerpapa commented on Sep 9, 2026

Deterministic Windows reproduction: 0.0 versus 0, without a crash or model call

Producer reproduction

Recovery comparison

Root cause and existing fixes

ChuLaiTheGreat commented on Sep 9, 2026

rdleclerc commented on Sep 10, 2026

Additional macOS occurrence: history still frozen after desktop update

Environment

User-visible impact

Exact failure boundary

Reproduction scope and unknowns

Requested resolution

Jok98 commented on Sep 10, 2026

Environment

Observed failure

Controlled recovery evidence

gabriel-andreescu commented on Sep 10, 2026

Two Windows recovery failures after unclean shutdowns on 0.153.4

Environment

Case A: duplicate ordinal blocks projection and forking

Case B: zero-filled region and a checkpoint inside a valid record

Fix state on `main` (verified against current tags)

Deterministic Windows reproduction: `0.0` versus `0`, without a crash or model call