Repository navigation
codex-cli stalls for exactly 300s between tool result and next model request (stream disconnected) #23807
Description
What version of Codex CLI is running?
codex-cli 0.132.0
What subscription do you have?
Plus
Which model were you using?
gpt-5.5
What platform is your computer?
Microsoft Windows NT 10.0.26100.0 x64
What terminal emulator and version are you using (if applicable)?
VS Code integrated terminal (PowerShell), codex running via openai.chatgpt VS Code extension
Codex doctor report
{
"schemaVersion": 1,
"generatedAt": "1779332610s since unix epoch",
"overallStatus": "fail",
"codexVersion": "0.132.0",
"checks": {
"app_server.status": {
"id": "app_server.status",
"category": "app-server",
"status": "ok",
"summary": "background server is not running",
"details": {
"control socket": "C:\\Users\\zhouj\\.codex\\app-server-control\\app-server-control.sock",
"daemon state dir": "C:\\Users\\zhouj\\.codex\\app-server-daemon",
"mode": "ephemeral",
"pid file": "C:\\Users\\zhouj\\.codex\\app-server-daemon\\app-server.pid (missing)",
"settings": "C:\\Users\\zhouj\\.codex\\app-server-daemon\\settings.json (missing)",
"status": "not running",
"update-loop pid file": "C:\\Users\\zhouj\\.codex\\app-server-daemon\\app-server-updater.pid (missing)"
},
"remediation": null,
"durationMs": 0
},
"auth.credentials": {
"id": "auth.credentials",
"category": "auth",
"status": "ok",
"summary": "auth is configured",
"details": {
"auth file": "C:\\Users\\zhouj\\.codex\\auth.json",
"auth storage mode": "File",
"stored API key": "true",
"stored ChatGPT tokens": "false",
"stored agent identity": "false",
"stored auth mode": "api_key"
},
"remediation": null,
"durationMs": 0
},
"config.load": {
"id": "config.load",
"category": "config",
"status": "ok",
"summary": "config loaded",
"details": {
"CODEX_HOME": "C:\\Users\\zhouj\\.codex",
"config.toml": "C:\\Users\\zhouj\\.codex\\config.toml",
"config.toml parse": "ok",
"enabled feature flags": "shell_tool, shell_snapshot, terminal_resize_reflow, sqlite, hooks, enable_request_compression, multi_agent, apps, tool_search, tool_suggest, plugins, plugin_hooks, in_app_browser, browser_use, browser_use_external, computer_use, plugin_sharing, image_generation, skill_mcp_dependency_install, steer, guardian_approval, goals, collaboration_modes, tool_call_mcp_elicitation, personality, fast_mode, tui_app_server, workspace_dependencies",
"feature flag overrides": "goals=true",
"feature flags enabled": "28",
"log dir": "C:\\Users\\zhouj\\.codex\\log",
"mcp servers": "0",
"model": "gpt-5.5",
"model provider": "custom",
"sqlite home": "C:\\Users\\zhouj\\.codex"
},
"remediation": null,
"durationMs": 0
},
"installation": {
"id": "installation",
"category": "install",
"status": "ok",
"summary": "installation looks consistent",
"details": {
"PATH codex #1": "C:\\Users\\zhouj\\AppData\\Roaming\\npm\\codex",
"PATH codex #2": "C:\\Users\\zhouj\\AppData\\Roaming\\npm\\codex.cmd",
"PATH codex entries": "2",
"current executable": "C:\\Users\\zhouj\\AppData\\Roaming\\npm\\node_modules\\@openai\\codex\\node_modules\\@openai\\codex-win32-x64\\vendor\\x86_64-pc-windows-msvc\\codex\\codex.exe",
"install context": "npm",
"managed by bun": "false",
"managed by npm": "true",
"managed package root": "C:\\Users\\zhouj\\AppData\\Roaming\\npm\\node_modules\\@openai\\codex",
"npm update target": "C:\\Users\\zhouj\\AppData\\Roaming\\npm\\node_modules\\@openai\\codex"
},
"remediation": null,
"durationMs": 358
},
"mcp.config": {
"id": "mcp.config",
"category": "mcp",
"status": "ok",
"summary": "no MCP servers configured",
"details": {},
"remediation": null,
"durationMs": 0
},
"network.env": {
"id": "network.env",
"category": "network",
"status": "ok",
"summary": "network-related environment looks readable",
"details": {
"proxy env vars present": "NO_PROXY, no_proxy"
},
"remediation": null,
"durationMs": 0
}
}
}What issue are you seeing?
Codex CLI periodically stalls for exactly 300 seconds (5 minutes) during a turn, showing "Working (Xm Ys)" with no network activity. After the 300s timeout, it logs stream disconnected - retrying sampling request (1/5) and resumes normally. This happens multiple times per session (5 occurrences across ~15 hours), making long-running agentic tasks unreliable.
From TUI log (~/.codex/log/codex-tui.log):
Instance 1 — exact 300.003s gap:
2026-05-20T12:21:44.542667Z INFO codex_core::client: close time.busy=10.9ms time.idle=40.2s
*** 300 seconds of silence — no client:new, no ToolCall ***
2026-05-20T12:26:44.545840Z WARN codex_core::session::turn: stream disconnected - retrying sampling request (1/5 in 197ms)...
2026-05-20T12:26:44.746956Z INFO codex_core::client: new <- retry succeeds immediately
Instance 2 — exact 300.021s gap:
2026-05-20T13:12:50.025022Z INFO codex_core::client: close time.busy=25.9ms time.idle=58.0s
*** 300 seconds of silence ***
2026-05-20T13:17:50.046571Z WARN codex_core::session::turn: stream disconnected - retrying sampling request (1/5 in 218ms)...
2026-05-20T13:17:50.267074Z INFO codex_core::client: new <- retry succeeds immediately
All 5 occurrences in this session:
- 2026-05-20T12:26:44 (retry in 197ms)
- 2026-05-20T13:17:50 (retry in 218ms)
- 2026-05-20T15:42:53 (retry in 210ms)
- 2026-05-20T15:52:04 (retry in 180ms)
- 2026-05-21T00:56:48 (retry in 196ms)
The retry always succeeds in under 250ms, proving the network path is healthy. The stall is entirely internal to codex_core.
Additional context:
app-server event consumer lagged; dropping ignored events skipped=93/135/231appears in the same session- The same local proxy (127.0.0.1:15721) handles Claude Code traffic without any stalls
- Average
time.idleacross all 583 requests in this session is 85s (normal model thinking time) - Large
time.idlevalues up to 1262s occur during active HTTP streams WITHOUT triggering "stream disconnected", confirming the 300s watchdog only fires between requests
What steps can reproduce the bug?
-
Configure codex with a custom provider endpoint:
model_provider = "custom" model = "gpt-5.5" model_reasoning_effort = "xhigh" sandbox_mode = "danger-full-access" [model_providers.custom] name = "custom" wire_api = "responses" supports_websockets = false requires_openai_auth = true base_url = "http://127.0.0.1:15721/v1"
-
Start a codex session with a complex task that requires multiple tool calls (file reads, shell commands, code edits)
-
Let it run for 30+ minutes
-
Observe periodic 300-second gaps where no HTTP requests are sent to the model endpoint, followed by "stream disconnected - retrying sampling request (1/5)"
Detailed sequence before first stall:
12:20:40 client: close (model returned tool_call response, time.idle=29.5s)
12:20:40 ToolCall: shell_command (read test_supervisor.py)
12:20:40 ToolCall: shell_command (read test_session_adapter.py)
12:20:40 ToolCall: shell_command (git diff)
12:21:03 client: close (model responded, time.idle=22.4s)
12:21:04 ToolCall: shell_command (Get-Content SKILL.md) <- executes in ~200ms
12:21:04 client: new <- sends tool result back to model
12:21:44 client: close time.idle=40.2s <- model responds normally
*** 300 seconds of silence — no client:new, no ToolCall ***
12:26:44 stream disconnected - retrying (1/5 in 197ms)
12:26:44 client: new <- retry works instantly
The model's response at 12:21:44 completes normally. After client: close, codex_core should either issue the next client: new or complete the turn. Instead, nothing happens for exactly 300 seconds.
No thread ID available (custom provider endpoint).
What is the expected behavior?
After client: close, the turn should immediately either:
- Dispatch the next tool call from the response, or
- Send the next
client: newrequest with tool results, or - Complete the turn
There should be no 300-second idle gap between a successful response and the next action. The 300s value is suspiciously precise (300.003s and 300.021s), suggesting a hardcoded timeout constant in codex_core::session::turn that fires when the turn state machine fails to advance after receiving a complete response.
Additional information
Analysis:
The 300-second value appears to be a hardcoded stream-activity watchdog in codex_core::session::turn. The pattern suggests a race condition where the turn state machine sometimes fails to advance after receiving a complete model response. It sits idle until the watchdog fires and forces a retry.
This is NOT a network issue because:
- The same local proxy handles Claude Code traffic without any stalls
- The retry after 300s succeeds in <250ms every time
time.idlevalues on normal requests (22-134s) show the model endpoint is responsive
Possibly related:
app-server event consumer lagged; dropping ignored events skipped=93/135/231— TUI event loop under pressure, may indicate backpressure blocking the turn from advancing- Multiple
codex_core::tools::router: error=Exit code: 1errors in the same session (tool execution failures) model_reasoning_effort = "xhigh"produces longer model responses and more tool calls per turn, possibly increasing the chance of hitting the race condition
Suggested investigation areas:
- The 300s constant in
codex_core::session::turn— is this a stream keepalive/watchdog timer? - Possible race condition between the response parser completing and the turn state machine's next-step dispatch
- Whether
app-server event consumer laggedindicates backpressure that blocks the turn from advancing - Whether the
responseswire API path has a different state machine than thechatpath
Log grep patterns for reproduction:
grep "stream disconnected" ~/.codex/log/codex-tui.log
grep "client: close\|client: new" ~/.codex/log/codex-tui.log
grep "app-server event consumer lagged" ~/.codex/log/codex-tui.log
Session stats:
Log spans: 2026-05-05 to 2026-05-21
Total client new/close pairs: 1255
Stream disconnects: 5
Average time.idle: 85s
Max time.idle (without triggering disconnect): 5463s
Activity
github-actions commentedon May 21, 2026
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
jshaofa-ui commentedon May 21, 2026
Summary
Issue: codex-cli 0.132.0 stalls for exactly 300 seconds (5 minutes) between completing a tool execution and sending the next model request. This occurs on Windows with VS Code integrated terminal (PowerShell), using gpt-5.5 model with Plus subscription.
Root Cause: The Responses API WebSocket connection is reused across multiple model requests within a single turn. When the WebSocket connection becomes stale or dead (due to network issues, proxy timeouts, server-side disconnection, or Windows-specific networking behavior), the client does not detect this until it sends the next request and waits for the first event. The
stream_idle_timeoutdefault of 300,000ms (300s = 5 minutes) is used as the timeout for waiting for the first event after sending a request, causing the exact 300-second stall observed.Fix Approach: Introduce a separate
first_event_timeout(default: 60s) for waiting on the first event after sending a WebSocket request, distinct from the existingstream_idle_timeout(default: 300s) used for subsequent events during streaming. Add WebSocket connection health detection via ping/pong keepalive. This ensures dead connections are detected within 60s rather than 300s, with automatic reconnection on the next request attempt.Root Cause Analysis
Architecture Overview
The codex-cli turn execution follows this flow:
The 300-Second Timeout
The default stream idle timeout is defined in
model-provider-info/src/lib.rs:This timeout is applied in two places in the WebSocket path:
send_websocket_request()- Timeout for sending the request frame:run_websocket_response_stream()- Timeout for waiting for each event:Why the Stall Occurs
The stall sequence:
run_sampling_request()calledstream_request()called on existing connectionsend_websocket_request()→ TCP send may succeed (data buffered locally) or fail silentlyrun_websocket_response_stream()→ Waits for first event with 300s timeoutCodexErr::Stream("idle timeout waiting for websocket", None)Why Windows/VS Code is Affed
Several factors make this more likely on Windows with VS Code integrated terminal:
TCP Keepalive Differences: Windows TCP keepalive defaults (2 hours) are much longer than Linux (75 seconds), so dead connections aren't detected at the TCP level.
Proxy/Load Balancer Behavior: Corporate networks on Windows often route through proxies that may silently drop idle WebSocket connections after a timeout period.
VS Code Terminal Multiplexing: The integrated terminal adds an additional layer of I/O handling that can mask connection state changes.
PowerShell I/O Handling: PowerShell's handling of stdout/stderr streams may buffer or delay error propagation.
Network Profile Changes: Windows network profile changes (WiFi ↔ Ethernet, VPN connect/disconnect) can break connections without proper TCP FIN/RST.
Why HTTP Path Doesn't Exhibit This Issue
The HTTP/SSE path creates a new connection for each request, so a dead connection is detected immediately during TCP connect (connection refused), not after waiting 300s. The WebSocket path reuses the same connection, so dead connections are only detected when data is sent and no response arrives.
Proposed Fix
Approach
The fix has three components:
Separate First-Event Timeout: Introduce
first_event_timeout(default: 60s) for waiting on the first event after sending a WebSocket request. This is distinct fromstream_idle_timeout(300s) which remains for subsequent events during streaming.WebSocket Ping/Pong Keepalive: Implement periodic ping frames to detect dead connections proactively, before the next request is sent.
Connection Health Check: Before reusing a WebSocket connection, perform a lightweight health check.
Code Changes
1. Add
first_event_timeoutto Provider ConfigurationFile:
codex-rs/model-provider-info/src/lib.rs2. Update Provider Struct
File:
codex-rs/codex-api/src/provider.rs3. Update WebSocket Connection to Use First-Event Timeout
File:
codex-rs/codex-api/src/endpoint/responses_websocket.rs4. Update
run_websocket_response_streamwith Separate TimeoutsFile:
codex-rs/codex-api/src/endpoint/responses_websocket.rs5. Update WebSocket Client Connection to Pass First-Event Timeout
File:
codex-rs/codex-api/src/endpoint/responses_websocket.rs6. Add WebSocket Ping/Pong Keepalive (Optional but Recommended)
File:
codex-rs/codex-api/src/endpoint/responses_websocket.rs7. Update Config Schema
File: Update config schema to include new field
The
stream_idle_timeout_msconfig option already exists. Addfirst_event_timeout_ms:Testing Strategy
Unit Tests
First-Event Timeout Test:
Dead Connection Detection Test:
Keepalive Ping Test:
Integration Tests
End-to-End Tool Execution Test:
Windows-Specific Test:
Long-Running Turn Test:
Regression Tests
Normal Operation Test:
HTTP Fallback Test:
Performance Tests
Timeout Impact Measurement:
Keepalive Overhead:
Impact Assessment
Positive Impacts
first_event_timeout_msbased on their network conditionsRisk Assessment
Edge Cases
Configuration Recommendations
For users experiencing connectivity issues:
For users on stable connections (no change needed):
Files Modified
codex-rs/model-provider-info/src/lib.rsfirst_event_timeout_msfield, constant, and methodcodex-rs/codex-api/src/provider.rsfirst_event_timeouttoProviderstructcodex-rs/codex-api/src/endpoint/responses_websocket.rscodex-rs/codex-api/src/lib.rsfirst_event_timeout_msdocumentationAlternative Approaches Considered
1. Reduce
stream_idle_timeoutGloballyRejected: Would cause false positives on long streaming responses (e.g., complex reasoning that produces events infrequently).
2. TCP Keepalive Configuration
Rejected: TCP keepalive is OS-level and affects all connections. Not configurable per-connection in reqwest/tungstenite easily. Also, TCP keepalive defaults are too long on Windows.
3. Connection Pool with Health Checks
Rejected: Adds significant complexity. The current single-connection-per-turn design is simpler and the first-event timeout approach achieves the same goal with less complexity.
4. Always Create New WebSocket Connection
Rejected: Would lose the benefits of connection reuse (sticky routing, previous_response_id caching, reduced latency). The keepalive + first-event timeout approach maintains connection reuse while detecting dead connections.
Conclusion
The 300-second stall is caused by the WebSocket connection reuse pattern combined with the 300-second default idle timeout. When the connection becomes stale or dead between model requests, the client waits the full 300 seconds before detecting the failure.
The proposed fix introduces a separate 60-second timeout for the first event after sending a request, reducing the stall from 300s to 60s in the worst case. Combined with WebSocket ping/pong keepalive, dead connections can be detected proactively, often before the next request is even sent.
This fix is backward compatible, low-risk, and addresses the root cause without changing the fundamental architecture of connection reuse.
Avishayy commentedon May 27, 2026
We're hitting what looks like the same issue on codex-cli 0.133.0, and can add a couple of data points from packet + app-server traces that may help localize it.
Same signature: periodic stalls where a turn hangs after dispatch with no output, then recovers cleanly. Our stalls that we let run to natural recovery come back at ~300s, matching the
exact-300s timing reported here. Independent of context size, reasoning effort, or rate limits (rate limits were ~1% when it happened).
Two differences/additions from this report:
The model client span shows the idle gap directly:
model_client.stream_responses_websocket{model=gpt-5.5 wire_api=responses
transport="responses_websocket" ...}: codex_core::client: close
time.busy=3.74ms time.idle=304s
i.e. the connection was open and idle for 304s with the turn awaiting frames, then closed.
2. Packet capture shows at least one concrete trigger: a stale pooled connection that the server RSTs, which codex doesn't fast-fail. At the instant the turn dispatched, codex pushed
the request onto a pooled connection the server had already closed and got an immediate RST:
So it's not (in this case) backend silence or a dead path — the server actively reset a reused idle connection, and codex appears to wait the full ~300s idle window before surfacing it
as stream disconnected - retrying instead of fast-failing on the RST. The retry itself works fine and recovers in <250ms once it fires — consistent with this report. So the painful
part is the ~300s detection latency, not the retry.
Questions for maintainers:
pooled-connection RST trigger an immediate retry independent of the idle timeout?
Environment: codex-cli 0.133.0, app-server mode (driven via JSON-RPC), gpt-5.5, model_reasoning_effort=high, Linux. Happy to share a sanitized pcap excerpt or full codex_core::client /
tungstenite frame trace if useful.
rebroad commentedon Aug 6, 2026
Adding one complementary point to the first-event-timeout/keepalive proposal above:
The retry policy should also have a bounded aggregate budget, rather than allowing each of the 5 stream retries to consume the full 300-second idle timeout. Otherwise one stale connection can produce 25+ minutes of apparent “Working” (and reports show 30–45 minute cases).
Possible design:
This should complement fast handling of TCP resets and ping/pong health checks: the transport can recover quickly when possible, but a repeatedly unhealthy path cannot leave the turn looking indefinitely active.
HomCrazyEveryDay commentedon Aug 12, 2026
I had the same issue.
Reconnecting 2/5
stream disconnected before completion:
idle timeout waiting for websocket
300.003/300.021ms