Skip to content

Active goal auto-resume after an unclean restart leaves interrupted lifecycle state and diverges from persisted goal status after context-full retries (autogen selfreport from codex) #32973

Open

Description

@nos1609

What version of Codex CLI is running?

codex-cli 0.144.3 was the failing runtime.

0.144.4 was the latest release when this report was filed. I did not retry the original damaged goal on 0.144.4 because it is retained as forensic evidence and another resume would mutate its state.

The affected thread was originally created by codex-cli 0.142.5 and continued across CLI updates.

What subscription do you have?

Pro

Which model were you using?

gpt-5.6-sol, reasoning effort ultra

What platform is your computer?

Linux x86_64, kernel 6.12, RHEL-family distribution

What terminal emulator and version are you using (if applicable)?

VS Code with Codex extension build 26.707.71524, backed by local app-server processes. A codex-tui 0.144.3 client was also used during diagnosis.

Codex doctor report

{
  "status": "not available",
  "reason": "codex doctor --json did not produce a report; a bounded retry timed out"
}

What issue are you seeing?

An active long-running goal did not recover safely after an unclean host restart. I am not asserting that Codex caused the restart.

This is the observed sequence:

  1. The goal was active and a root continuation plus one child worker were in flight.
  2. The host restarted without a clean shutdown.
  3. After VS Code reopened the same thread, the extension reported resumeState=needs_resume and latestTurnStatus=interrupted, then became the stream owner.
  4. Automatic goal continuation started while persisted state still contained the interrupted root turn and stale descendant/controller records. I found no durable reconciliation event before that continuation.
  5. Several automatic goal turns ran over approximately 15 minutes.
  6. Every attempt completed without reasoning, tools, or an assistant response and had last_agent_message=null.
  7. The final attempts had zero last-token usage and matching turn-specific app-server errors:
Codex ran out of room in the model's context window. Start a new thread or clear earlier history before retrying.

Sanitized event summary:

Attempt Offset from first resumed turn Assistant/tool items last_agent_message Last-token usage Runtime result
1 +0 none null terminal zero record after an inherited usage record no retained turn error
2 about +25 seconds none null 0 no retained turn error
3 about +3 minutes none null 0 context-window error
4 about +13 minutes none null 0 context-window error; goal DB later read blocked

The persisted sequence was effectively:

thread resume: latest turn = interrupted
thread_goal_updated: status = active
task_started
internal goal continuation input
token_count: last usage = 0
task_complete: last_agent_message = null
...
thread_goals SQLite row: status = blocked

After the final empty completion, the goal DB row was blocked and its persisted update timestamp matched that completion. The latest durable rollout goal event still said active. No matching blocked event or machine-readable failure reason was present in the rollout.

The context was large but valid JSONL: approximately 230 MB, approximately 105k records, and more than 100 persisted compaction events. The rollout remained parseable and SQLite integrity checks passed. I found no evidence of file corruption, but I did not have a pre-restart snapshot that could rule out missing records.

Post-restart inspection also found stale lifecycle state. Most of it existed before the restart, so it is supporting context for missing recovery cleanup, not evidence that the restart created it:

  • multiple root task_started events had no task_complete or turn_aborted;
  • several direct child edges remained open even though most corresponding child rollouts had completed;
  • one child worker was genuinely interrupted by the restart;
  • several agent-job controller rows remained running although their items had completed or were never assigned.

The observations point to a goal recovery/reconciliation or status-propagation failure; they do not establish which component is the root cause.

The user-visible result is an unrecoverable goal that appears to auto-continue, produces empty completed turns, emits no useful conversation error, and becomes blocked only in the goal database.

What steps can reproduce the bug?

I do not yet have a deterministic minimal reproduction. The following is a reproduction hypothesis derived from one observed incident; use only in an isolated environment:

  1. Use an isolated CODEX_HOME and a disposable repository.
  2. Start a persistent active goal in a tool-heavy thread.
  3. Let the thread approach its context limit and keep a goal continuation or child worker in flight.
  4. Terminate the owning app-server without pausing the goal, simulating a process crash or unclean machine restart.
  5. Start a fresh app-server or reopen the thread from VS Code.
  6. Capture whether the following observed failure signature occurs:
    • resume identifies the latest turn as interrupted;
    • automatic goal continuation starts while persisted interrupted/stale lifecycle state remains;
    • the internal goal continuation is injected into context-full history;
    • turns complete with last_agent_message=null and zero usage instead of a visible error;
    • automatic retries repeat;
    • the goal DB becomes blocked while the latest durable rollout goal event remains active.

What is the expected behavior?

After an unclean process or host restart, active-goal recovery should be transactional and explicit:

  • In-flight root turns and descendants should be reconciled to interrupted or failed terminal states.
  • The goal should be paused or blocked before automatic continuation if ownership or context state is uncertain.
  • If automatic continuation is safe, it should run only after context validation or successful compaction.
  • A context-window failure must be emitted to the client and rollout as a visible error, not represented as a successful completed turn with a null assistant message.
  • Goal status should remain consistent between the rollout event stream and the goal database.
  • A blocked goal should persist a machine-readable reason and offer a supported recovery action, such as starting a new thread from a compact goal handoff.

Additional information

Related but not duplicate:

Uploaded diagnostic thread ID: 019f43ac-20b8-7ad1-965b-3b2e6460255d

The thread was uploaded through the Codex report/feedback flow so maintainers can correlate it with server-side diagnostics. Raw prompts, absolute paths, host name, project or repository names, timezone, process IDs, and raw rollout/log files remain omitted because they contain private information. More exact event counts and schema-level evidence can be provided privately if maintainers need them.

This issue was generated by Codex from a local read-only forensic review at the GitHub user's request. The user explicitly authorized publication after private identifiers were removed.

Activity

  1. added
    bugSomething isn't working
    extensionIssues related to the VS Code extension
    app-serverIssues involving app server protocol or interfaces
    sessionIssues involving session (thread) management, resuming, forking, naming, archiving
    contextIssues related to context management (including compaction)
    subagentIssues involving subagents or multi-agent features
    on Jul 14, 2026
  2. nos1609 commented on Jul 14, 2026

    @nos1609
    Author

    Recovery update: successful but unsupported workaround

    I recovered uploaded thread 019f43ac-20b8-7ad1-965b-3b2e6460255d without starting a new thread. The same goal is active again and normal reasoning/tool calls are continuing.

    This also refines the diagnosis:

    • Both SQLite databases passed PRAGMA integrity_check, and the full rollout remained valid JSONL. I found no torn database or JSONL write.
    • The last pre-restart request was already at 237,789 / 258,400 tokens. On resume, context reconstruction reached the model limit; the empty retries then ended in the reported context-window error.
    • The goal DB transition to blocked was not random corruption: it was the terminal-error guard stopping another retry loop. The unresolved product issue is the unsafe resume/reconciliation path, the silent empty turns, and the missing matching durable blocked reason/event.
    • The rollout contained two interleaved compaction chains and 18 persisted open child edges that no longer existed in the runtime registry.

    Recovery procedure used:

    1. Quiesce Codex UI/app-server writers and verify that no process has the target rollout open for append.

    2. Preserve complete backups first. Use SQLite's online backup API for live logical database snapshots; do not naively copy individual database/WAL/SHM files while writers are active. Copy the complete rollout and record hashes.

    3. Work only on copies in an isolated CODEX_HOME.

    4. Select the latest complete compaction checkpoint bundle, not only the compacted record. In this uploaded rollout the tested bundle was lines 104551-104555:

      compacted
      world_state (full)
      turn_context
      token_count
      context_compacted
      

      Its post-compaction token state was 43,344 / 258,400.

    5. Retain the rollout prefix through context_compacted, validate every JSONL record, and resume the copy while leaving the goal blocked. The copied thread hydrated as idle and did not modify the candidate rollout.

    6. Atomically replace the live rollout only while no writer has it open. Reopen the thread without resuming the goal and run one bounded control turn. That turn completed normally at 46,284 / 258,400.

    7. Reconcile stale descendants. In this case close_agent returned agent ... not found, while state_5.sqlite.thread_spawn_edges still contained 18 open rows for the parent. After another database backup, those independently verified ghost edges were changed to closed in one parent-scoped SQLite transaction. Post-change: open = 0, PRAGMA integrity_check = ok.

    8. Resume through the supported /goal resume flow. Do not force thread_goals.status = 'active' with SQL.

    Result:

    • goal status: active
    • normal reasoning and tool calls resumed
    • observed context after continuation: 58,459 / 258,400
    • persisted open child edges: 0
    • the context-window error did not recur

    This modifies undocumented internal storage and is not a supported user-facing fix. Record layouts and schemas may change between releases. Do not attempt it without complete backups, quiesced writers, and isolated-copy validation. A direct goal-status edit alone does not repair oversized reconstructed context.

    A built-in recovery action could automate the safe parts: roll back to the last complete compaction bundle, reconcile interrupted/stale descendants, persist a machine-readable blocked reason, show the discarded suffix, and only then offer goal resume.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app-serverIssues involving app server protocol or interfacesbugSomething isn't workingcontextIssues related to context management (including compaction)extensionIssues related to the VS Code extensionsessionIssues involving session (thread) management, resuming, forking, naming, archivingsubagentIssues involving subagents or multi-agent features

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions