Skip to content

High-reasoning Responses WebSocket idle timeout is a false dead-stream (model resumed after 8.5 min of no text frames) #39771

Open
Open
High-reasoning Responses WebSocket idle timeout is a false dead-stream (model resumed after 8.5 min of no text frames)#39771
Labels
appIssues related to the Codex desktop appbugSomething isn't workingconnectivityIssues involving networking or endpoint connectivity problems (disconnections)

Description

@aygerix
aygerix
opened on Aug 20, 2026

What version of the Codex App are you using (From “About Codex” dialog)?

ChatGPT desktop 26.814.41407 (bundled codex-cli 0.148.0-alpha.15)

What subscription do you have?

ChatGPT Pro (plan_type=pro in local rate-limit snapshots). Auth mode: ChatGPT (not API key).

What platform is your computer?

Darwin 24.6.0 arm64 arm
macOS 15.7.7 (24G720)

What issue are you seeing?

On gpt-5.6-sol with reasoning_effort=high, a turn that is still generating can go several minutes with no Responses text frames. The client treats that silence as a dead websocket:

WARN codex_core::responses_retry
stream disconnected - retrying sampling request (n/5) ...
sampling_error=stream disconnected before completion: idle timeout waiting for websocket

The UI stays active with frozen token counts. There is no user-visible disconnect. After 5 silent retries the turn is wasted. Interrupt shows turn_aborted.

This is not (only) a dropped TCP connection. Protocol ping/pong do not reset the idle timer (they are swallowed in the websocket pump and never reach ws_stream.next()). The watchdog is tokio::time::timeout(stream_idle_timeout, ws_stream.next()) with default stream_idle_timeout_ms = 300_000. Only a text frame resets it.

stream_idle_timeout_ms / supports_websockets cannot be changed for the built-in openai provider (model_providers.openai is reserved / or_insert ignores overrides).

What steps can reproduce the bug?

  1. Codex Desktop, ChatGPT login, model gpt-5.6-sol, reasoning high.
  2. Run a long turn: many exec calls, then a long think/write (large patch / gate file). Transport is wss://chatgpt.com/backend-api/codex/responses.
  3. After the last commentary + tool result, the model goes quiet. No agent_reasoning / tool / token snapshot advances for >5 minutes.
  4. Client logs idle timeout waiting for websocket and retries the entire sampling request (1/5 … 5/5). Chat still looks “working”.

Controlled experiment (same machine, same thread, same model): a localhost reverse proxy in front of https://chatgpt.com/backend-api/codex injected a text frame every 45s while upstream was silent:

{"type":"codex.client.keepalive"}

Codex accepted the frames and ignored them as unknown events:

DEBUG codex_api::sse::responses
unhandled responses event: "codex.client.keepalive"

No idle timeout fired. After ~8.5 minutes of no real model output (18:06:20Z → 18:14:56Z on 2026-08-20), the same in-flight sampling request emitted a real custom_tool_call / patch_apply_end and continued. Frozen token counts then moved.

That is the opposite of a dead stream: the websocket and the server were still in the turn; the 300s text-frame watchdog was a false positive.

Earlier incident on the same thread without the keepalive (same day):

UTC What happened
17:00:02 Last real commentary (“now freezing the … gate”)
17:07:34 / 17:13:11 / 18:18:50 idle timeout waiting for websocket retries 1/5, 2/5, 3/5
17:21:49 User interrupt; turn_aborted; promised file never written

Thread id: 01a02010-e5b0-75f1-b3b7-3e0c54879f59
Turn (keepalive run): 01a02059-3cbe-70d2-9c1b-881ae7f1cfa4
Context at last good request: ~99k / 258400 input tokens (~38%). rate_limit_reached_type was null (weekly usage was high, ~95%, but not marked reached).

Same idle timeout waiting for websocket pattern on other gpt-5.6-sol high/medium Desktop turns the same day (websocket closed by server before response.completed and Connection reset without closing handshake also occur, but those are distinct).

What is the expected behavior?

  • Do not treat in-progress high-reasoning silence (no Responses text frames, websocket still open) as a dead stream.
  • Server or client should emit a liveness text/event that the idle waiter counts, or the idle timer should not apply until the socket is actually closed / ping fails.
  • If you keep a 300s watchdog: surface it in the UI (“waiting for model, last frame Ns ago”) instead of silent retrying sampling request (n/5) with frozen tokens.
  • Allow stream_idle_timeout_ms (and/or websocket opt-out) on the built-in ChatGPT openai provider. Today the only documented knobs are on custom providers.

A faster dead-stream detector (#38638, #17003) would have killed this turn at 5 minutes. The model resumed at 8.5 minutes. Faster detection of true RSTs is still good; collapsing “no text frames” into “dead” is not.

Additional information

Related: #38638, #27625, #28579, #24260, #23807, #26696, #33051.

#23807 is a 300s stall between completed responses (turn state machine). This report is idle timeout during an open sampling stream while gpt-5.6-sol high is thinking and sending no text.

#38638 matches the log line and the “cannot override built-in provider” constraint; the keepalive experiment is extra evidence that the 300s timeout is often a false dead-stream, not only slow detection of a real drop.

Code references on main (behavior matches 0.148.0-alpha.15):

  • DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000 in codex-rs/model-provider-info/src/lib.rs
  • timeout(idle_timeout, ws_stream.next()) → "idle timeout waiting for websocket" in codex-rs/codex-api/src/endpoint/responses_websocket.rs
  • Ping/Pong handled inside the pump and not forwarded to next()
  • Unknown JSON events: parse failure or unhandled responses event → continue (timer already reset)

Activity

  1. github-actions
    added
    bugSomething isn't working
    appIssues related to the Codex desktop app
    connectivityIssues involving networking or endpoint connectivity problems (disconnections)
    on Aug 20, 2026

  2. github-actions commented on Aug 20, 2026

    @github-actions
    github-actionsbot
    on Aug 20, 2026 – with GitHub Actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  3. aygerix commented on Aug 20, 2026

    @aygerix
    aygerix
    on Aug 20, 2026
    Author

    Not a duplicate of #38638 — same log line, opposite diagnosis.

    #38638 treats idle timeout waiting for websocket as a dead stream that takes 300s to notice, and asks for faster detection / HTTPS fallback on the first retry.

    This issue is that the same 300s watchdog is a false dead-stream during in-progress high-reasoning generation. The websocket is still open; the model is just not emitting Responses text frames. Protocol ping/pong do not reset the timer.

    Evidence from a controlled keepalive: injecting {"type":"codex.client.keepalive"} (client logs unhandled responses event: "codex.client.keepalive") prevented the idle timeout. The same in-flight sampling request then produced a real tool/patch after ~8.5 minutes of no model text (18:06:20Z → 18:14:56Z). Faster detection as requested in #38638 would have killed that turn at 5 minutes.

    Please keep this open (or fold the keepalive result into #38638 if you prefer one thread). The user-visible hang is shared; the fix is not “detect death faster,” it is “do not treat high-reasoning silence as death,” plus a UI signal instead of silent retrying sampling request (n/5).

  4. aygerix commented on Aug 20, 2026

    @aygerix
    aygerix
    on Aug 20, 2026
    Author

    openai/codex does not accept external PRs (docs/contributing.md), so this is a proposed patch for the team rather than a pull request.

    What to change

    The 300s watchdog is on application text frames, not websocket liveness.

    In codex-rs/codex-api/src/endpoint/responses_websocket.rs:

    • WsStream answers Ping with Pong and drops both Ping and Pong. They never reach ws_stream.next().
    • run_websocket_response_stream does timeout(idle_timeout, ws_stream.next()) with idle_timeout = provider.stream_idle_timeout (default 300_000 ms).
    • Result: a live socket with no Responses text for 5 minutes (normal for gpt-5.6-sol high) becomes idle timeout waiting for websocket, then silent retrying sampling request (n/5).

    A localhost proxy that injected {"type":"codex.client.keepalive"} every 45s (unknown event → unhandled responses event, continue) stopped the timeout. The same in-flight request then emitted a real tool/patch after 8.5 minutes of no model text. So the stream was not dead.

    Suggested fix (do not just raise 300s)

    Split two clocks:

    1. Transport liveness (seconds): send a WebSocket Ping on an interval; treat Ping/Pong as activity; fail only if ping/write fails or the socket closes. That also satisfies Responses WebSocket stalls 300s per retry; opt-out flag removed; HTTPS fallback requires 5 failed retries #38638 / Detect dead websocket connections faster after network changes instead of showing Working for up to 5 minutes #17003 (detect real death faster than 5 minutes).
    2. Application silence during an in-flight response.create: do not retry the whole sampling request just because no output_item arrived for 300s. Optional: UI “last frame Ns ago”. Optional safety cap near the existing 60-minute websocket_connection_limit_reached.

    Do not only bump DEFAULT_STREAM_IDLE_TIMEOUT_MS. That makes true RSTs slower to notice.

    Minimal code shape in the existing pump:

    Ok(Message::Ping(payload)) => {
    if inner.send(Message::Pong(payload)).await.is_err() { break; }
    let _ = tx_message.send(Ok(Message::Ping(Default::default()))); // reset idle waiter
}
Ok(Message::Pong(_)) => {
    let _ = tx_message.send(Ok(Message::Pong(Default::default())));
}

    and in the read loop, Ping/Pong → continue (already almost there, but those messages never arrive today). Plus a heartbeat Send { Message::Ping(..) } every ~15–20s.

    Config gap

    [model_providers.openai] stream_idle_timeout_ms = … is ignored: merge_configured_model_providers uses or_insert for openai. Docs document the key, but it cannot be set for ChatGPT Desktop. Allowing timeout/retry overrides on the built-in id would at least let users opt into a longer application wait without a proxy. That is a workaround, not a substitute for (1)+(2).

    Happy to clarify logs from thread 01a02010-e5b0-75f1-b3b7-3e0c54879f59 / turn 01a02059-3cbe-70d2-9c1b-881ae7f1cfa4 if useful.

  5. AlexADittrich commented on Sep 3, 2026

    @AlexADittrich
    AlexADittrich
    on Sep 3, 2026

    Additional Windows data from a separate longitudinal run (2026-08-27 through 2026-09-03) supports the same false-dead-stream diagnosis:

    • Codex CLI 0.151.0-alpha.7.2 was tested; 0.153.0 and ChatGPT Desktop 26.901.20858 were also present on the same machine.
    • Windows 11 Pro 10.0.26200, ChatGPT auth (not API key), wired Ethernet, no proxy or VPN.
    • Across 65,733 local log rows: 427 occurrences of idle timeout waiting for websocket; zero occurrences for idle timeout waiting for SSE.
    • The failures began abruptly on 2026-08-27 after zero observed failures on 2026-08-26.
    • For 184 first-failure events, p10/p50/p90 gaps from the last received stream event were 300.0s / 300.2s / 306.5s; 183 of 184 were at least 300s.
    • Retry frequency increased with reasoning effort: effectively absent at low, 1.17 events/turn at medium, 1.71 at high, 3.10 at xhigh, and 5.25 at ultra.
    • 23 of 191 failure bursts exhausted all five retries; recovered bursts still re-sent the full request.

    This also weakens a local network explanation: reconnects commonly succeeded in about one second with HTTP 101 and immediately produced reasoning output. The practical user-side configuration remains unavailable because the built-in openai provider cannot be overridden, while the legacy WebSocket feature flags are removed.

    Suggested fixes remain: count protocol liveness/keepalives separately from application text-frame progress, avoid retrying an in-progress response solely because no text frame arrived for 300s, and expose a supported timeout override for the built-in provider.

  6. doodead
    added a commit that references this issue on Sep 8, 2026

    Fix false Responses WebSocket idle timeout

    8ef6915

  7. mcip3301 commented on Sep 12, 2026

    @mcip3301
    mcip3301
    on Sep 12, 2026 – with ChatGPT Codex Connector
    No description provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    appIssues related to the Codex desktop appbugSomething isn't workingconnectivityIssues involving networking or endpoint connectivity problems (disconnections)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions