Skip to content

Serialize concurrent rollout writers before assigning paginated ordinals #40109

Open

Description

@bjornfix

Two independent RolloutRecorder instances can resume the same paginated rollout, each read the same tail ordinal, and append duplicate ordinals. The rollout projection then fails with an ordinal mismatch even though the JSONL remains syntactically valid.

This patch fixes the owning writer seam by taking an OS-level exclusive lock for each append batch, refreshing ordinal state from the durable tail while locked, and applying the same protection to the cold append path. It also adds a regression test that resumes two recorders and asserts the resulting ordinals are contiguous and unique.

The patch contains no rollout contents, credentials, or account-specific configuration.

Activity

  1. added
    bugSomething isn't working
    CLIIssues related to the Codex CLI
    sessionIssues involving session (thread) management, resuming, forking, naming, archiving
    on Aug 22, 2026
  2. github-actions commented on Aug 22, 2026

    @github-actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  3. pomazanbohdan commented on Aug 23, 2026

    @pomazanbohdan

    This writer-race hypothesis matches a Windows/Desktop incident where a reboot/reopen boundary produced token_count and thread_settings_applied with the same ordinal. A safe cleaner must require full writer absence, preserve the canonical suffix, and fail closed on any unknown duplicate; see #35746 and #40178.

  4. AsusFarstrider commented on Aug 28, 2026

    @AsusFarstrider

    I can confirm the same duplicate-ordinal / wedged-history failure mode on Linux using the Codex CLI/TUI, with a different trigger than the VS Code ownership-transfer case in #38629.

    Environment:

    • Linux
    • Codex CLI 0.150.0
    • paginated local thread history
    • model: gpt-5.6-sol

    Trigger:
    An unexpected disconnect occurred during a long-running CLI/TUI thread. I was not opening the same conversation in multiple VS Code windows or intentionally running concurrent turns.

    After resuming:

    • The canonical rollout JSONL still contains the later conversation/work.
    • The rollout contains a duplicate ordinal: 1754.
    • The history projector repeatedly reports:
      expected ordinal 1755, got 1754
    • The indexed/materialized history stops at that boundary.
    • An older turn beginning with "Please continue" remains marked inProgress.
    • codex resume therefore displays history only up to that stale boundary.
    • Codex itself can still read/use the later rollout context and correctly knows about work completed after the visible cutoff.
    • The supported migration command reports the thread as already migrated/paginated and does not repair the projection.
    • I did not manually modify the rollout or Codex's SQLite databases.

    This appears to be the same underlying duplicate-ordinal / history-projection failure described here, but the trigger in this case was an unexpected disconnect in the Linux CLI/TUI rather than the multi-window VS Code ownership race in #38629.

    It also closely resembles the stale-history behavior in #41079 on Windows Desktop: the durable rollout remains complete while the paginated projection becomes permanently stuck behind a duplicate ordinal.

    A supported reconciliation/repair path would be useful in addition to preventing duplicate ordinals, since once this state exists the thread remains usable by Codex internally but its visible resumed history is permanently stale.

  5. MichaelSpece commented on Sep 1, 2026

    @MichaelSpece

    Correction / follow-up (2026-09-01): A controlled single-process resume reproduced the duplicate ordinal, so the competing-writer inference in my original comment was incorrect for this instance.

    The root cause matches the decoder asymmetry documented in #40630:

    • The projection accepts each JSONL line as serde_json::Value and then uses decode_rollout_line.
    • Resume-tail ordinal discovery parsed directly into RolloutLine.
    • With serde_json/arbitrary_precision, that typed path can reject a flattened token_count record containing a floating-point used_percent.
    • The reverse scan then steps back one record and reissues the skipped tail ordinal.

    An isolated regression with ordinals [0, 1, token_count@2] reproduced [0, 1, 2, 2] on append. Changing resume-tail discovery to scan serde_json::Value and call decode_rollout_line produced [0, 1, 2, 3]. The full CLI/TUI binary built successfully, and a clean single-process resume no longer introduced a collision.

    The already-damaged thread was also recoverable in this specific case: after a private byte-for-byte backup, I removed only the redundant duplicate boundary record, preserved the remaining suffix, and resumed with the fixed binary. The rollout is now strictly contiguous and the SQLite projection checkpoint reaches EOF.

    Environment: Linux CLI/TUI, codex-cli 0.149.0. Thread IDs, paths, transcript contents, and raw databases are intentionally omitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIbugSomething isn't workingsessionIssues involving session (thread) management, resuming, forking, naming, archiving

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions