Repository navigation
[Bug] Competing paginated rollout writers emit overlapping ordinal sequences #41353
Description
Summary
Paginated rollout JSONL can receive overlapping ordinal sequences from competing/stale writer state. A single duplicate or backward ordinal then permanently wedges thread_history_projection_state, leaving the UI on an old snapshot while the canonical transcript continues growing.
This is broader than a one-off corrupted cursor: multiple active threads can become affected simultaneously, and one rollout can contain several independent ordinal regressions.
Related symptom report: #41079.
Environment
- Codex Desktop on Windows 11 x64
- Paginated thread history
- Observed with a current bundled alpha runtime
Sanitized fleet evidence
- All SQLite databases passed
PRAGMA quick_check,foreign_key_check, and migration checks. - Four live threads were wedged concurrently.
- Logs contained more than 1,200 repeated
failed to project durable rolloutwarnings. - Ten non-monotonic ordinal transitions were found across the 43 rollouts represented in
thread_history_projection_state. - One affected rollout contained three separate regressions, so advancing the SQLite cursor past only the first duplicate would fail again later.
Producer pattern
A representative sequence was:
ordinal N: event_msg/token_count
ordinal N+1: event_msg/token_count
ordinal N: event_msg/thread_settings_applied
ordinal N+1: event_msg/thread_goal_updated
ordinal N+2: event_msg/task_started
ordinal N+3: later durable history continues normally
This looks like two append paths assigning ordinals from independently cached tail state. The newer turn starts from stale N even though another writer has already persisted N and N+1.
Projection failure
read_projection_steps currently treats every ordinal < next_ordinal as fatal. Consequently, even stale records that produce no projected turns/items permanently block all later valid history:
thread history projection expected ordinal N+2, got N
The durable rollout remains complete and parseable. The SQLite view simply stops advancing.
Expected behavior
- All writes to one rollout must share one serialized ordinal allocator, or refresh and lock the durable tail before assigning ordinals.
- A stale decoded record that is provably a thread-history no-op must not permanently wedge projection.
- Meaningful duplicate or out-of-order records must still fail closed or trigger a safe rebuild rather than being silently discarded.
- Projection failure should be surfaced in the UI and have an automatic repair path.
Tested defensive patch
Patch and regression test:
The patch skips a stale ordinal only when:
- the line decoded successfully;
- no rejected lines precede it;
project_rollout_lineproduces no changed turns or items;- it is not a projected realtime item.
Meaningful duplicate ordinals remain rejected by the existing test.
Verification:
cargo test -p codex-thread-store
235 passed; 0 failed
The regression test was verified red before the implementation (expected ordinal 2, got 1) and green afterward.
Privacy
No thread identifiers, prompts, absolute paths, credentials, raw logs, or user content are included.
Activity
github-actions commentedon Aug 28, 2026
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
Gavin-Qiao commentedon Aug 29, 2026
Maintainer invitation request
I independently reproduced this failure class on another installation: the canonical paginated rollout remained parseable and complete, SQLite integrity checks passed, and projection stopped at a stale duplicate metadata ordinal while later valid records continued. No user content or identifiers are needed to reproduce it.
If invited, I can prepare an external PR with the narrow defensive scope described in this issue:
ordinal < next_ordinal, skip the record only if it decoded cleanly, no rejected-line gap precedes it, projection produces no changed turns or items, and it is not a projected realtime item.The serialized-writer / durable-tail refresh remains the separate root-cause prevention work.
Per the repository's contribution policy, I will not open an unsolicited PR. Would a maintainer please confirm whether this boundary aligns with the intended solution and explicitly invite the PR if so?
harukifu commentedon Aug 31, 2026
Additional Windows reproduction on 26.825.6671.0
I reproduced the same failure class on Codex for Windows 26.825.6671.0, Windows 11 x64 build 26200, locale ja-JP.
User-visible symptom: in a very long active thread, the conversation view repeatedly jumps or returns to older messages while new assistant/tool output continues to be appended. It appears as though the chat has rolled back and makes the latest state difficult to follow.
Sanitized local evidence:
codex_thread_store::local::live_writerthread history projection expected ordinal 28715, got 28714thread-store/src/local/live_writer.rsAn additional clue is that the long-running session referenced the previous package version 26.818.5229.0 and then continued under 26.825.6671.0, so restoring/continuing a very long active thread across an app update may be relevant.
This is an observed correlation with the visible jump-back behavior; I am not claiming proven causation. The thread/turn identifiers, raw SQLite database, transcript, local paths, credentials, and full logs are intentionally omitted from this public comment. Precise identifiers can be supplied through a private OpenAI feedback channel if needed.