[Windows][Desktop] Duplicate token_count/task_started ordinal permanently freezes thread history projection #44888

What version of the Codex App are you using (From “About Codex” dialog)?

26.903.71938

What subscription do you have?

ChatGPT Pro 20x

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What issue are you seeing?

One specific long-running Codex Desktop session repeatedly reopened at a much older fixed point in the conversation after restarting Codex Desktop.

The strange part was that the session itself was still alive: if I sent another message, the agent could continue the latest work correctly. Other sessions behaved normally.

There was no user-visible error message in the Desktop UI.

I inspected the underlying session data and confirmed that the canonical rollout JSONL was still intact, still growing, and contained the recent conversation that was missing from the Desktop UI.

The rollout was approximately 280.7 MB.

The thread-history projection, however, was stuck very early in the rollout:

next_rollout_byte_offset = 19935796
next_rollout_ordinal     = 1715

At exactly that byte offset, the rollout contained a duplicated top-level ordinal:

offset 19933888 -> 19934893   ordinal 1713   response_item / custom_tool_call_output
offset 19934893 -> 19935796   ordinal 1714   event_msg / token_count
offset 19935796 -> 19936044   ordinal 1714   event_msg / task_started
offset 19936044 -> 19936190   ordinal 1715   world_state
offset 19936190 -> 19937002   ordinal 1716   turn_context

So the projection expected ordinal 1715, but the record at its stored byte offset was another 1714.

The SQLite database itself was healthy:

PRAGMA integrity_check = ok

This appears to be a durable-history projection failure rather than transcript deletion or general SQLite corruption.

Before recovery, the affected thread had only:

thread_history_projection_state = 1 row
thread_items                     = 580 rows
thread_turns                     = 2 rows

while the rollout contained much more recent history.

What steps can reproduce the bug?

I do not currently have a deterministic minimal reproduction, but this is the observed sequence:

1. Use a long-running Codex Desktop session over many turns.

2. Continue using and restarting/resuming the session normally.

3. At some point, the rollout contains two consecutive top-level records with the same ordinal:

   ordinal 1714   event_msg / token_count
   ordinal 1714   event_msg / task_started
   ordinal 1715   world_state
   

4. Restart Codex Desktop.

5. Open the affected session.

6. The visible conversation history always stops at the same old point.

7. Send a new message to the session.

8. The agent can still continue the latest work, indicating that the canonical session/context has not actually been lost.

9. Restart Codex Desktop again.

10. The UI again falls back to the same stale history prefix.

Affected session ID:

01a04d3f-d35e-7881-8463-18d8c2d12794

The session rollout was located under the normal Windows Codex sessions directory and continued growing after the UI history had stalled.

At the time of diagnosis, the latest rollout records had reached approximately ordinal 100926, while the durable history projection was still stuck around ordinal 1715.

I did not record the exact token-limit/context-window values at the original failure boundary.

A read-only way to detect the failure is to compare:

SELECT
    thread_id,
    next_rollout_byte_offset,
    next_rollout_ordinal
FROM thread_history_projection_state
WHERE thread_id = ?;

against the actual JSONL record located at next_rollout_byte_offset.

In my case:

projection expected: ordinal 1715
actual record:       ordinal 1714

because ordinal 1714 appeared twice consecutively.

What is the expected behavior?

Every top-level rollout record should have a unique, monotonically increasing ordinal.

A duplicate or replayed ordinal should not permanently prevent all subsequent valid conversation history from being materialized.

If the canonical rollout remains intact but the derived thread-history projection becomes inconsistent, Codex should automatically recover or rebuild the affected projection.

After restarting Codex Desktop, the full latest conversation history should remain visible rather than reverting to an old fixed point.

Ideally, Codex should also expose a supported repair/reindex mechanism for an affected thread instead of requiring users to inspect or modify thread_history_1.sqlite manually.

Additional information

I was able to recover the session completely without modifying the canonical rollout JSONL.

Before making any changes, I:

• closed all Codex processes;
• backed up the rollout JSONL;
• backed up thread_history_1.sqlite, including an SQLite-native backup;
• verified PRAGMA integrity_check = ok.

I first tried a minimal cursor correction:

next_rollout_ordinal: 1715 -> 1714

while keeping:

next_rollout_byte_offset = 19935796

unchanged.

The update succeeded, but reopening Codex Desktop did not trigger the projection to advance. It remained at:

(19935796, 1714)

The recovery that worked was:

1. Close all Codex processes.

2. Keep the canonical rollout JSONL untouched.

3. Delete only the derived rows belonging to this single affected thread from:

   thread_history_projection_state
   thread_items
   thread_turns
   

4. Run:

   codex resume 01a04d3f-d35e-7881-8463-18d8c2d12794
   

Codex then rebuilt the history projection from the intact rollout.

After rebuilding:

Projection byte offset = 280740279
Rollout file length    = 280740279
next_rollout_ordinal   = 100926
thread_items           = 31076
thread_turns           = 2820

The projection byte offset exactly matched the rollout file length.

After exiting the CLI and reopening Codex Desktop, the latest conversation history was visible normally again.

So, in this case:

canonical rollout remained intact
        ↓
duplicate ordinal occurred
        ↓
derived history projection permanently stalled
        ↓
live session could still continue
        ↓
Desktop restart showed only the stale prefix
        ↓
rebuilding the per-thread derived projection from the intact rollout restored the complete history

This appears related to existing reports such as #43262, #42027, and #44609.

The important additional observation from this incident is that simply rewinding the stored projection ordinal did not cause Codex Desktop to resume materialization; removing the affected thread's derived projection and then running codex resume did successfully rebuild it to EOF.

I have retained local backups of the original rollout and the pre-repair SQLite database, but I have not attached them because they contain private conversation/project data. I can provide additional sanitized diagnostics if needed.