app-server: silent turn/completed(last_agent_message=null) when run_turn early-returns after compaction failure #25619

What issue are you seeing?

Several early-return paths in codex-rs/core/src/session/turn.rs::run_turn make the function return None (which yields a task_complete with last_agent_message=null, zero last_token_usage, and no time_to_first_token_ms), without emitting an EventMsg::Error to the wire/rollout. The companion emit_turn_error_lifecycle call in those paths only notifies in-process extension contributors; tracing::error!() only writes to the local log. JSON-RPC clients connected to a codex app-server --listen see a normal-looking completed turn and cannot tell that the model was never sampled.

From a programmatic client's view, the WebSocket exchange is:

-> turn/start { threadId, input: [...] }
<- turn/started
<- token/count   (total_token_usage.total_tokens == model_context_window if set_token_usage_full has fired previously)
<- token/count
<- turn/completed { turn: { id, status: "completed" },
                    last_agent_message: null,
                    time_to_first_token_ms: null,
                    duration_ms: 2_000-12_000 }

No error event. No item events. Indistinguishable on the wire from a normal completion in which the model voluntarily returned an empty assistant message.

Affected paths (HEAD checkout)

In codex-rs/core/src/session/turn.rs::run_turn:

• The pre-sampling compact branch around L147-L162 (catching run_pre_sampling_compact Err) calls emit_turn_error_lifecycle + tracing::error!("Failed to run pre-sampling compact") then return None;. It does not call sess.send_event(turn_context, EventMsg::Error(...)).
• The mid-turn auto-compact branch (auto-compact after a sampling result, ~L290-L316) follows the same pattern: extension lifecycle + tracing log + return None;, no wire-visible event.

By contrast, the existing catch-all at the bottom of the main loop (~L394-L411) does emit the wire-visible event:

Err(e) => {
    info!("Turn error: {e:#}");
    let error = e.to_codex_protocol_error();
    sess.emit_turn_error_lifecycle(turn_context.as_ref(), error.clone()).await;
    if error == CodexErrorInfo::UsageLimitExceeded { /* ... */ }
    let event = EventMsg::Error(e.to_error_event(/*message_prefix*/ None));
    sess.send_event(&turn_context, event).await;
    break;
}

And codex-rs/core/src/compact.rs:242-244 follows the same convention before returning Err(ContextWindowExceeded). So the established pattern across the codebase is "emit EventMsg::Error before failing the turn" — the two paths above just slipped through.

What steps can reproduce the bug?

1. Bring up a codex app-server --listen unix:///tmp/codex.sock whose persistent session is in the "context-full" state, i.e. one of its prior turns triggered Session::set_total_tokens_full (either via the ContextWindowExceeded branch in try_run_sampling_request at core/src/session/turn.rs:985-988, or via auto-compact exhaustion in core/src/compact.rs:242). After that fires once, state.history.token_info is pinned at total_tokens == model_context_window until a successful compaction resets it.
2. From any JSON-RPC client (e.g. a small Python WebSocket script or websocat), drive:
   • initialize
   • thread/resume { "threadId": "<id>", "excludeTurns": true, "persistExtendedHistory": true }
   • turn/start { "threadId": "<id>", "input": [{ "type": "text", "text": "<<no reply needed>>", "text_elements": [] }], "approvalPolicy": "on-request" }
3. Read frames until turn/completed for the new turn_id.

Observed: turn/completed with turn.status = "completed", last_agent_message = null, all-zero last_token_usage, no error event in between. The rollout JSONL records task_started, turn_context, the response_item for the user message, the user_message event, two token_count events (with the synthetic total_token_usage.total_tokens == model_context_window), then task_complete. Nothing else.

Observation that surfaced this

On a long-lived persistent session (open for several weeks, ~85k rollout lines, ~138MB):

• Several consecutive turn/start requests to the app-server over multiple hours produced silent task_complete(null) turns (durations 2.1 s - 11.4 s, no ttft, no Error).
• Recovery happened only when a user typed input into a separate codex resume --last process attached to the same thread. That second process has its own in-memory Session and apparently completed a compaction successfully; subsequent turns through it sampled normally.
• The app-server's in-memory session state remained context-full for the entire failing window. Rate limits were fine (primary 2%, secondary 13%, rate_limit_reached_type: null).

What is the expected behavior?

When run_turn is going to return None because of a pre-sampling or mid-turn compaction failure (or any path where the model was not actually invoked), the app-server should emit a wire-visible EventMsg::Error (or a new variant such as EventMsg::TurnFailed) before the task_complete, so a JSON-RPC client can detect that the turn was not sampled. The task_complete itself could additionally carry a richer status than "completed" in those cases (e.g. "aborted"/"failed"/"context_full"), but even the simpler fix — propagating the existing EventMsg::Error — would close the observability gap.

Minimal illustrative patch

Aligns the two missing paths to the convention already used at turn.rs:394-411 and compact.rs:242-244:

// core/src/session/turn.rs - pre-sampling compact branch
if let Err(err) = run_pre_sampling_compact(&sess, &turn_context, &mut client_session).await {
    let error = err.to_codex_protocol_error();
    sess.emit_turn_error_lifecycle(turn_context.as_ref(), error.clone()).await;
    if error == CodexErrorInfo::UsageLimitExceeded
        && let Err(e2) = sess.goal_runtime_apply(GoalRuntimeEvent::UsageLimitReached {
            turn_context: turn_context.as_ref(),
        }).await
    {
        warn!("failed to usage-limit active goal after usage-limit error: {e2}");
    }
+   let event = EventMsg::Error(err.to_error_event(/*message_prefix*/ None));
+   sess.send_event(&turn_context, event).await;
    error!("Failed to run pre-sampling compact");
    return None;
}

And the analogous insertion in the mid-turn run_auto_compact failure branch a few hundred lines lower. Both should adopt the e.to_codex_protocol_error() + e.to_error_event(None) pair already in use at L394-L411 (ownership of err allows both calls, same as the existing pattern).

Additional information

• Why this is a bug and not a feature: extension contributors and tracing logs are not a substitute for the JSON-RPC surface. Any client that consumes codex app-server programmatically — IDE integrations, codex exec, our JSON-RPC dispatcher — has no robust way today to distinguish "model produced an empty response" from "model was never invoked". The current behavior also disagrees with itself: the catch-all path at L394 emits EventMsg::Error; the two paths above don't. Two-thirds of the same handler family emits an observable failure signal; one-third silently swallows it.
• Related issues (similar symptom, different surface or root cause):
  • codex exec can silently complete empty when configured MCP tools are deferred behind tool_search #24536 - codex exec can silently complete empty when configured MCP tools are deferred behind tool_search (same observable from a different cause; a generic wire-level failure signal would help diagnostics there too).
  • app-server websocket shared-thread turns complete but rollout never materializes, so sibling thread/resume fails with "no rollout found" #16872 - app-server websocket shared-thread turns complete but rollout never materializes (adjacent symptom in the same area).
  • Running out of room in the Codex context window is immediately fatal to that chat thread #7808 - Running out of room in the Codex context window is immediately fatal to that chat thread (the human-facing UX of the same underlying state).
• Environment:
  • Codex binary: codex 0.125.0 was the persistent app-server at the time of the observation (running as a systemd unit, codex app-server --listen unix:///run/.../app.sock); the interactive sibling was codex 0.134.0 via codex resume --last. Source examined: HEAD of https://github.com/openai/codex.git.
  • OS: Linux (Debian, Linux 6.12.85+deb13-amd64).
  • Protocol used by the client: v2 JSON-RPC over Unix-socket WebSocket (the client does a manual HTTP Upgrade handshake).
• Investigation context: this was found while hardening our JSON-RPC dispatcher against the symptom. The full investigation notes (with the per-turn event sequences and code-level tracing) are local; happy to attach a sanitized excerpt if useful. The client-side fix is independent (it will start to declare processed=false based on last_agent_message, last_token_usage, item-event count, and the very EventMsg::Error this issue proposes to emit), but a wire-level signal from Codex would let every programmatic client benefit, not just ours.
• Not asking for: a redesign of the context-full state machine. Even keeping the current pinned token_info after set_total_tokens_full, just emitting EventMsg::Error (or equivalent) on the early-return paths would already make this diagnosable and correctable from the client side.