Repository navigation
HTTP SSE response.failed waits for EOF and can be replaced by idle timeout (0.153.4; tested fix) #43140
Description
Summary
The HTTP SSE consumer in Codex CLI 0.153.4 does not terminate when it parses a terminal response.failed event. It saves the error and keeps reading until EOF. If the server keeps the connection open, the client waits for its stream idle timeout, then reports idle timeout waiting for SSE instead of the original server error. A subsequent transport error can also overwrite the terminal error.
This is a deterministic, provider-independent client bug, reproduced against the original installed binary using a loopback HTTP server with no model service or credentials. A narrow source fix and four regression tests are included below, together with baseline and patched results.
I investigated this after repeated reconnecting/stalls with an Azure Responses deployment. I have not established that this exact terminal-event/open-socket sequence caused the live Azure reconnects. Real backend failures were also observed outside Codex. Please treat this issue as a confirmed SSE failure-handling defect, not a claim that a client patch repairs Azure capacity.
What Version of Codex CLI Is Running?
- Original installed executable:
codex-cli 0.153.4. - Official source tag used for the patch:
rust-v0.153.4, commit3d2ee51ca2d5db578f328aa75e20aa22c0197c9a. - The latest stable release returned by GitHub when checked on September 6, 2026 (UTC) was
rust-v0.153.4, published September 4, 2026 at 23:25:48 UTC. - Desktop version:
26.901.41600, build7982. - The inspected
mainsnapshot at commit008bbd5884122dc95aaece19ecfe0fc6a59dcf36also has the store-and-wait behavior incodex-api/src/sse/responses.rs(lines 663-676). The proposed diff is pinned to the release above, not to movingmain.
Subscription, Model, Platform, and Terminal
- Provider: user-configured Azure OpenAI Responses endpoint, API-key authentication; the failing requests are not routed through a ChatGPT subscription.
- Model/deployment identifier:
gpt-6-astra; response headers identifiedgpt-6-astra-2026-09-03, West US, pay-as-you-go. These are identifiers observed in this deployment, not a claim about public model availability. - Reasoning effort:
medium; configured context window:1000000; auto-compact threshold:272000. - OS: macOS 26.6.2, build 25G83; Darwin 25.6.0; Apple Silicon arm64.
- Original symptom: Codex Desktop. Deterministic reproducer: noninteractive
codex exec --jsonwith piped stdio, no terminal emulator required.
Sanitized effective provider settings:
model = "gpt-6-astra"
model_provider = "azure"
model_reasoning_effort = "medium"
model_context_window = 1000000
model_auto_compact_token_limit = 272000
[model_providers.azure]
name = "Azure Responses"
base_url = "https://<resource>.openai.azure.com/openai/v1"
wire_api = "responses"
requires_openai_auth = false
env_http_headers = { "api-key" = "AZURE_OPENAI_API_KEY" }
request_max_retries = 4
stream_max_retries = 10
stream_idle_timeout_ms = 300000The fixture below overrides retries to zero and the idle timeout to 800 ms in a separate CODEX_HOME. It does not alter any live configuration. The real configuration also explicitly disables remote_compaction_v2. No model, context, output limit, retry, or timeout change is part of the SSE fix.
What Issue Are You Seeing?
The server has already declared the response failed. Waiting for transport closure after that declaration delays reporting/retry and can discard the useful error classification/message/retry delay. With the production timeout above, a hold-open connection can incur a 300-second idle wait; the reproduction uses 800 ms so this behavior can be tested quickly.
Relevant release source: codex-rs/codex-api/src/sse/responses.rs, process_sse_with_treatment.
The old error branch assigns response_error = Some(error.into_api_error()). EOF returns that saved value, but the timeout and transport-error branches return different errors before consulting it.
Steps to Reproduce
- Run the Python 3 standard-library script below against an original 0.153.4 CLI. No Azure/OpenAI access, credentials, catalog, or real model inference is needed.
- The script starts a loopback HTTP SSE fixture and uses a fresh isolated
CODEX_HOMEfor each case. Both cases send HTTP 200,Content-Type: text/event-stream, and the same terminal failure:
event: response.failed
data: {"type":"response.failed","response":{"id":"resp_local_terminal_repro","status":"failed","error":{"code":"server_error","message":"LOCAL_TERMINAL_FAILURE_MARKER"}}}
- First case closes after the event; second case leaves the socket open. Read
summary.jsonand the CLI error events.
python3 codex_sse_terminal_repro.py --codex /absolute/path/to/codex --output ./repro-original
# Repeat against the patched binary with a new output directory:
python3 codex_sse_terminal_repro.py --codex /absolute/path/to/patched/codex --output ./repro-patchedComplete standalone reproducer (Python standard library only)
#!/usr/bin/env python3
"""Exercise an installed Codex against a local SSE failure; never calls Azure."""
import argparse
import asyncio
from datetime import datetime, timezone
import json
import os
from pathlib import Path
import shutil
import time
MARKER = 'LOCAL_TERMINAL_FAILURE_MARKER'
async def run_case(args, name, hold_open):
root = args.output / name
root.mkdir(parents=True, exist_ok=False)
home = root / 'home'
home.mkdir()
requests = []
handlers = set()
started = time.monotonic()
started_at_utc = datetime.now(timezone.utc).isoformat()
async def handler(reader, writer):
task = asyncio.current_task()
handlers.add(task)
try:
raw_headers = await reader.readuntil(b'\r\n\r\n')
headers = {}
for line in raw_headers.decode().split('\r\n')[1:]:
if ':' in line:
key, value = line.split(':', 1)
headers[key.lower()] = value.strip()
size = int(headers.get('content-length', '0'))
await reader.readexactly(size)
event = {'type': 'response.failed', 'response': {
'id': 'resp_local_terminal_repro', 'status': 'failed',
'error': {'code': 'server_error', 'message': MARKER}}}
body = ('event: response.failed\ndata: ' + json.dumps(event) + '\n\n').encode()
writer.write(b'HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nConnection: close\r\n\r\n' + body)
await writer.drain()
requests.append({'sent_failure_after_s': round(time.monotonic() - started, 3),
'request_body_bytes': size})
if hold_open:
# Model a server that sends a terminal event without closing its socket.
try:
await asyncio.wait_for(reader.read(), timeout=8)
except asyncio.TimeoutError:
pass
finally:
writer.close()
await writer.wait_closed()
handlers.discard(task)
server = await asyncio.start_server(handler, '127.0.0.1', 0)
port = server.sockets[0].getsockname()[1]
config = [
'model = ' + json.dumps(args.model),
'model_provider = "local_sse_fixture"',
'model_reasoning_effort = "medium"',
]
if args.catalog:
config.append('model_catalog_json = ' + json.dumps(str(args.catalog.resolve())))
config += [
'[model_providers.local_sse_fixture]',
'name = "Local SSE fixture only"',
f'base_url = "http://127.0.0.1:{port}/v1"',
'wire_api = "responses"',
'requires_openai_auth = false',
'request_max_retries = 0',
'stream_max_retries = 0',
f'stream_idle_timeout_ms = {args.idle_ms}',
]
(home / 'config.toml').write_text('\n'.join(config) + '\n')
process = None
try:
process = await asyncio.create_subprocess_exec(
args.codex, 'exec', '--skip-git-repo-check', '--json', '--sandbox', 'read-only',
'Reply only LOCAL_SSE_TEST. Do not call tools.',
cwd=root, env=dict(os.environ, CODEX_HOME=str(home)),
stdin=asyncio.subprocess.DEVNULL, stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE)
stdout, stderr = await asyncio.wait_for(process.communicate(), timeout=15)
elapsed = round(time.monotonic() - started, 3)
(root / 'stdout.jsonl').write_bytes(stdout)
(root / 'stderr.log').write_bytes(stderr)
events = []
for line in stdout.decode(errors='replace').splitlines():
try:
event = json.loads(line)
except json.JSONDecodeError:
continue
if event.get('type') in ('error', 'turn.failed'):
events.append(event)
serialized = json.dumps(events)
receipt = {'case': name, 'started_at_utc': started_at_utc,
'idle_timeout_ms': args.idle_ms, 'exit_code': process.returncode,
'elapsed_s': elapsed, 'request_count': len(requests),
'failure_to_exit_s': round(elapsed - requests[0]['sent_failure_after_s'], 3) if requests else None,
'original_error_preserved': MARKER in serialized,
'reported_idle_timeout': 'idle timeout' in serialized.lower(),
'events': events}
(root / 'receipt.json').write_text(json.dumps(receipt, indent=2))
print(json.dumps(receipt), flush=True)
return receipt
finally:
if process is not None and process.returncode is None:
process.terminate()
await process.wait()
server.close()
await server.wait_closed()
if handlers:
for task in list(handlers):
task.cancel()
await asyncio.gather(*list(handlers), return_exceptions=True)
async def main(args):
results = [await run_case(args, 'failed_then_eof', False),
await run_case(args, 'failed_then_open_socket', True)]
(args.output / 'summary.json').write_text(json.dumps(results, indent=2))
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--codex', default=shutil.which('codex') or '/Applications/ChatGPT.app/Contents/Resources/codex')
parser.add_argument('--output', type=Path, required=True)
parser.add_argument('--model', default='gpt-6-astra')
parser.add_argument('--catalog', type=Path)
parser.add_argument('--idle-ms', type=int, default=800)
args = parser.parse_args()
args.output = args.output.resolve()
args.output.mkdir(parents=True, exist_ok=True)
asyncio.run(main(args))Expected Behavior
Once a valid terminal response.failed has been classified, emit that error and stop consuming the stream immediately, independently of EOF. Preserve prior output/metadata, the existing error type and retry hint. Do not emit a later completion. Leave retry policy to the existing caller.
Actual Results and Verification
All durations below are measured from sending the failed event to the CLI process exiting, not total process startup time. Small millisecond differences are not a performance benchmark.
| Executable | Server behavior | Failure to process exit | Original error preserved | Idle timeout reported |
|---|---|---|---|---|
| Original 0.153.4 | Close after failure | 11 ms | Yes | No |
| Original 0.153.4 | Keep socket open (800 ms idle limit) | 833 ms | No | Yes |
| Patched release CLI | Close after failure | 11 ms | Yes | No |
| Patched release CLI | Keep socket open (800 ms idle limit) | 10 ms | Yes | No |
- Four new regression tests against original logic: 0 passed, 4 failed.
- Same four tests after the patch: 4 passed, 0 failed.
- Entire
codex-api --libunit suite after the patch and formatting: 168 passed, 0 failed. git apply --checkplus application to a pristine copy and byte-for-byte comparison: passed.- Full optimized CLI build:
cargo build --locked --release -p codex-cli --bin codex, isolated Rust 1.95.0, official release profile. Passed in 973.48 seconds. - External dependency versions, source URLs and checksums were verified unchanged from the release lockfile. The tag's manifest version was 0.153.4 while path packages in the checked-in lockfile were 0.0.0; only local workspace package versions were refreshed with
cargo update --offline -p codex-api. That build-environment adjustment is not part of the proposed patch.
The new tests cover:
- Server error with a stream that stays pending indefinitely; terminal error arrives and producer exits without EOF.
- Rate-limit error on the same open stream; the parsed two-second retry hint is retained and the producer exits.
- A transport error after terminal failure cannot replace the original error.
- Preceding partial output is retained; a bogus completion after terminal failure is not consumed.
Exact test commands from codex-rs/, using Rust 1.95.0:
cargo test --locked -p codex-api --lib terminal_failed_response_ -- --nocapture
cargo test --locked -p codex-api --libProposed Fix
Capture whether the parsed event is response.failed before moving it into process_responses_event. If that terminal event yields an error, send its existing classified error and return. Preserve the old fallback for nonterminal parser/event errors.
This changes one production source file. It does not change response.incomplete, transport routing, tool dispatch, tool replay semantics, backoff, retry budgets, output caps, or context settings. It does not undo tool side effects already performed before a failure. The WebSocket path was reviewed separately and already returns on the corresponding error path.
Complete patch, including four regression tests
--- a/codex-rs/codex-api/src/sse/responses.rs
+++ b/codex-rs/codex-api/src/sse/responses.rs
@@ -656,6 +656,7 @@
return;
}
+ let is_terminal_failure = event.kind() == "response.failed";
match process_responses_event(event) {
Ok(Some(event)) => {
let is_completed = matches!(event, ResponseEvent::Completed { .. });
@@ -668,7 +669,13 @@
}
Ok(None) => {}
Err(error) => {
- response_error = Some(error.into_api_error());
+ let error = error.into_api_error();
+ // A failed response is terminal even if the server keeps the socket open.
+ if is_terminal_failure {
+ let _ = tx_event.send(Err(error)).await;
+ return;
+ }
+ response_error = Some(error);
}
};
}
@@ -1082,6 +1089,119 @@
}
other => panic!("unexpected event: {other:?}"),
}
+ }
+
+ #[tokio::test]
+ async fn terminal_failed_response_does_not_wait_for_eof() {
+ let event = json!({
+ "type": "response.failed",
+ "response": { "error": { "code": "server_error", "message": "upstream failed" } },
+ });
+ let body = Bytes::from(format!("event: response.failed\ndata: {event}\n\n"));
+ let stream = stream::iter(vec![Ok(body)]).chain(stream::pending());
+ let (tx, mut rx) = mpsc::channel(16);
+ let mut task = tokio::spawn(process_sse(
+ Box::pin(stream),
+ tx,
+ Duration::from_secs(300),
+ None,
+ ));
+
+ let result = timeout(Duration::from_secs(1), rx.recv()).await;
+ if result.is_err() {
+ task.abort();
+ }
+ assert_matches!(
+ result.expect("terminal failure waited for EOF"),
+ Some(Err(ApiError::Retryable { message, delay: None })) if message == "upstream failed"
+ );
+ let finished = timeout(Duration::from_secs(1), &mut task).await;
+ if finished.is_err() {
+ task.abort();
+ }
+ finished
+ .expect("SSE task did not release the stream")
+ .unwrap();
+ }
+
+ #[tokio::test]
+ async fn terminal_failed_response_preserves_rate_limit_without_eof() {
+ let event = json!({
+ "type": "response.failed",
+ "response": { "error": {
+ "code": "rate_limit_exceeded",
+ "message": "Rate limit reached. Please try again in 2s.",
+ } },
+ });
+ let body = Bytes::from(format!("event: response.failed\ndata: {event}\n\n"));
+ let stream = stream::iter(vec![Ok(body)]).chain(stream::pending());
+ let (tx, mut rx) = mpsc::channel(16);
+ let mut task = tokio::spawn(process_sse(
+ Box::pin(stream),
+ tx,
+ Duration::from_secs(300),
+ None,
+ ));
+
+ let result = timeout(Duration::from_secs(1), rx.recv()).await;
+ if result.is_err() {
+ task.abort();
+ }
+ assert_matches!(
+ result.expect("rate limit waited for EOF"),
+ Some(Err(ApiError::RateLimitExceeded { message, delay }))
+ if message == "Rate limit reached. Please try again in 2s."
+ && delay == Some(Duration::from_secs(2))
+ );
+ let finished = timeout(Duration::from_secs(1), &mut task).await;
+ if finished.is_err() {
+ task.abort();
+ }
+ finished
+ .expect("SSE task did not release the stream")
+ .unwrap();
+ }
+
+ #[tokio::test]
+ async fn terminal_failed_response_is_not_replaced_by_transport_error() {
+ let event = json!({
+ "type": "response.failed",
+ "response": { "error": { "code": "server_error", "message": "upstream failed" } },
+ });
+ let body = Bytes::from(format!("event: response.failed\ndata: {event}\n\n"));
+ let stream = stream::iter(vec![
+ Ok(body),
+ Err(TransportError::Network(
+ "connection reset afterwards".into(),
+ )),
+ ]);
+ let (tx, mut rx) = mpsc::channel(16);
+ process_sse(Box::pin(stream), tx, Duration::from_secs(300), None).await;
+
+ assert_matches!(
+ rx.recv().await,
+ Some(Err(ApiError::Retryable { message, delay: None })) if message == "upstream failed"
+ );
+ assert!(rx.recv().await.is_none());
+ }
+
+ #[tokio::test]
+ async fn terminal_failed_response_preserves_preceding_output_and_stops() {
+ let output = json!({ "type": "response.output_text.delta", "delta": "partial output" });
+ let failed = json!({
+ "type": "response.failed",
+ "response": { "error": { "code": "server_error", "message": "upstream failed" } },
+ });
+ let completed = json!({ "type": "response.completed", "response": { "id": "resp_late" } });
+ let body = format!("data: {output}\n\ndata: {failed}\n\ndata: {completed}\n\n");
+
+ let events = collect_events(&[body.as_bytes()]).await;
+ assert_eq!(events.len(), 2);
+ assert_matches!(&events[0], Ok(ResponseEvent::OutputTextDelta(text)) if text == "partial output");
+ assert_matches!(
+ &events[1],
+ Err(ApiError::Retryable { message, delay: None }) if message == "upstream failed"
+ );
}
#[tokio::test]Related Incident Context and Ruled-Out Hypotheses
These observations explain how the defect was found; they are not required for the local reproduction.
Compact Routing Was a Separate Problem
An Azure preview /responses/compact route returned HTTP 500 even for a two-message input, while ordinary Responses calls succeeded. Moving the provider to /openai/v1 made ordinary Responses and /responses/compact succeed; an actual Codex app-server tool -> compact -> continuation test passed. Both requests went to the configured Azure resource, not a hard-coded OpenAI endpoint. That routing correction was retained, but reconnects continued.
There Were Genuine Upstream Failures Too
Live logs contained generic server errors, no healthy upstream, response-body transport errors, and this explicit admission error:
The system is currently experiencing high demand and cannot process your request. Your request exceeds the maximum usage size allowed during peak load. For improved capacity reliability, consider switching to Provisioned Throughput.
Associated HTTP headers indicated 200/SSE and substantial remaining token/request quota; this does not establish Azure's internal admission policy. Several errors happened seconds after response headers, well below the configured idle timeout. Those timings do not support calling every reconnect an idle-timeout event.
A Python HTTP client bypassing Codex also received HTTP 200 followed by response.failed / server_error in 7.616 seconds. Request start: 2026-09-05T23:55:18.182529Z; request ID carried in the error text: 050e8820-ced4-4ebb-bcf4-379b1fd86cfb. Subsequent requests under the same broad conditions succeeded. This establishes at least one failure at the Azure response layer, not the internal backend root cause. Live arrival time of response.failed versus EOF was not captured precisely enough to attribute those incidents to this client bug.
Image Base64 Was Not Ordinary Context Text in the Audited Request
A request built from an isolated copy of the affected task contained 11 properly typed input_image items, with 4,647,490 data-URI characters. No image data URI or long base64-like string was found in ordinary text fields. Server input usage was 75,508 tokens with image placeholders versus 81,020 with the images. Thus the sampled request did not exhibit the suspected multi-million-character base64-as-text regression. This was a diagnostic request built from history, not a byte-for-byte capture of a failed live request.
Other Controls Were Inconclusive, Not Fixes
- Short requests using text, JSON function tools, plain custom tools and Lark custom tools completed on both Azure v1 and preview routes. Long-history controls and a nonstreaming control also completed.
- Natural-history request controls completed 4/4; captured-header/plain-header controls completed 3/3. A captured default-feature Azure request was uncompressed.
- Six parallel-tool-flag probes completed but generated only one custom call each, so they do not prove multi-call correctness.
- Output-cap comparisons were too small and inconsistent to establish a causal fix. No global output cap or context reduction was applied.
- The previous provider had a stream retry setting of 5 versus 10 in the Astra configuration, which changes retry-chain length, not why a request fails. Doctor also reported
unbounded_connection_retriesamong enabled features; the SSE patch does not change that policy or claim a finite overall retry bound. - The affected task resumed useful output before this source patch was installed. That recovery must not be attributed to the patch.
Prior local GPT-5.5/5.6 investigations had involved preview compact routing, bounded/redacted log output, a version-specific invalid tool-schema HTTP 400 workaround, and some model claims of unavailable tools despite successful calls. Those were checked rather than blindly reapplied. None establishes that today's HTTP 200/SSE failures have the same cause.
Codex Doctor Report
Privacy-reviewed excerpt from the original installed executable, taken before activating the patch. Full output includes private paths and unrelated local configuration, so those fields are omitted. Overall status was fail because this noninteractive process has TERM=dumb; that is not an SSE diagnosis.
{
"codexVersion": "0.153.4",
"overallStatus": "fail",
"checks": {
"app_server.status": {
"status": "warning",
"summary": "background server socket is stale or unreachable"
},
"auth.credentials": {
"status": "ok",
"summary": "OpenAI auth is not required for the active model provider"
},
"config.load": {
"status": "ok",
"summary": "config loaded"
},
"desktop.app.version": {
"status": "ok",
"summary": "the desktop application is installed"
},
"desktop.app_server.handshake": {
"status": "ok",
"summary": "no desktop app-server handshake was recorded"
},
"desktop.security.enforcement": {
"status": "ok",
"summary": "the desktop application passed available macos security assessments"
},
"git.environment": {
"status": "ok",
"summary": "git version 2.43.0"
},
"installation": {
"status": "ok",
"summary": "installation looks consistent"
},
"mcp.config": {
"status": "ok",
"summary": "MCP configuration is locally consistent"
},
"network.env": {
"status": "ok",
"summary": "network-related environment looks readable"
},
"network.provider_reachability": {
"status": "ok",
"summary": "active provider endpoints are reachable over HTTP"
},
"network.websocket_reachability": {
"status": "ok",
"summary": "Responses WebSocket is not enabled for the active provider"
},
"runtime.provenance": {
"status": "ok",
"summary": "running local build on macos-aarch64"
},
"runtime.search": {
"status": "ok",
"summary": "search is OK (system)"
},
"sandbox.helpers": {
"status": "ok",
"summary": "sandbox configuration is readable"
},
"security.endpoint": {
"status": "ok",
"summary": "no supported endpoint protection detected"
},
"state.paths": {
"status": "ok",
"summary": "state paths and databases are inspectable"
},
"state.rollout_db_parity": {
"status": "warning",
"summary": "rollout files and state DB thread inventory differ"
},
"system.disk": {
"status": "ok",
"summary": "sufficient free disk space (100.5 GiB)"
},
"system.environment": {
"status": "ok",
"summary": "OS language en-US"
},
"terminal.env": {
"status": "fail",
"summary": "TERM=dumb - colors and cursor control are disabled"
},
"terminal.title": {
"status": "ok",
"summary": "terminal title configured"
},
"updates.status": {
"status": "ok",
"summary": "update configuration is locally consistent"
}
}
}The preexisting stale/unreachable persistent-server socket and rollout/database inventory warning are included for completeness. The local reproducer uses a fresh CODEX_HOME and a separate codex exec process, so neither shared Desktop state nor daemon reuse is needed to reproduce this defect.
Local Deployment and Scope
The full patched release CLI is installed in a versioned user-local directory alongside the matching bundled Code Mode helper. The signed Desktop bundle and original binary were not modified; deep/strict app signature verification still passes. A guarded CODEX_CLI_PATH launcher is configured for the next Desktop launch and persisted by a user LaunchAgent. It falls back to the bundled CLI when the Desktop build changes. The fallback branch was tested.
The installed launcher and binary, not just a source-level test harness, passed the loopback reproduction above. They also passed a real Azure app-server tool -> remote compact -> tool test, with the test marker retained, Code Mode host enabled and observed running, gpt-6-astra, medium reasoning and the original 1M context configuration. Test interval: 2026-09-06T01:45:58.912353+00:00 to 2026-09-06T01:46:20.962310+00:00. No error notifications were recorded in that short smoke test. It is not a long-duration or multi-task reliability claim.
The still-running Desktop app was deliberately not terminated mid-investigation. The user must fully quit and reopen it; post-restart Desktop executable selection has not yet been verified. CLI/app-server functionality was verified directly through the same installed launcher. The override is reversible, and no model/provider/retry/context settings were changed by this installation.
Patched executable SHA-256: cf46450d3bde54feedae108eec0f6245b6969d4402b5b5f8a824d7c6fedd19da. This is a locally built, tested workaround, not an official signed OpenAI release or an upstream-accepted fix.
The issue intentionally includes the complete portable reproduction and patch rather than linking to private chat history. API keys, authorization headers, Azure resource hostname, private user paths, raw prompts, image payloads and local task IDs are omitted. Request correlation above is diagnostic metadata, not a credential.
I searched existing issues for response.failed with EOF/idle-timeout and the SSE function name. Related #4161 concerns parsing retry-delay text, not waiting for EOF after an already-classified terminal failure; I did not find a matching duplicate.
Activity
1320800521 commentedon Sep 12, 2026
Independent reproduction from XBSTACK on macOS confirms the same terminal-event handling shape on both codex-cli 0.153.4 (ChatGPT.app bundled binary) and the current stable 0.154.0.
Test setup: local loopback HTTP SSE server, isolated temporary CODEX_HOME, retries disabled, stream_idle_timeout_ms = 800, no model/API credential/external network.
Results:
Minimal independent fixture + version matrix:
https://github.com/xbstack/codex-response-failed-idle-timeout-repro
Detailed notes and diagnostic boundary:
https://www.xbstack.com/en/ai/tools-lab/codex-response-failed-idle-timeout-sse/?utm_source=github&utm_medium=referral&utm_campaign=codex_response_failed_idle_timeout&utm_content=issue_followup&ref=github
I am not treating the proposed source patch in this issue as a released fix yet; I can rerun the fixture against the first release that contains the fix.
SouthWinter commentedon Sep 14, 2026
I have a matching real-world reproduction on the first-party OpenAI provider through Codex Desktop's remote-SSH app-server.
Environment:
gpt-6-astraandgpt-5.6-solObserved error:
Error running remote compact task: stream disconnected before completion: idle timeout waiting for SSEThis is intermittent rather than deterministic. In the same long-lived thread I counted:
One
gpt-5.6-solfailure followed a reported token count of 157,617 at2026-09-14T07:30:55Z; the task completed with the error at07:47:26Z. Around the same period, the app-server log repeatedly reportedfailed to refresh available models: timeout waiting for child process to exit.Switching from Astra to 5.6 did not eliminate the issue. Restarting the remote app-server unloaded the failed in-memory state and allowed the thread to continue, but this was only a recovery step.
At failure time, the still-running app-server had retained an earlier 120K auto-compact threshold; after restart it loaded the configured 240K threshold. This explains why compaction triggered below 240K, but not why the stream ended with an idle timeout.
The visible error is consistent with this issue's failure-handling defect: a terminal upstream
response.failedmay be getting masked by the later SSE idle timeout. I cannot confirm the original terminal event because the local rollout/log retained only the final idle-timeout message. Unlike the Azure setup in the original report, this case uses the first-party OpenAI/ChatGPT route with remote compaction v2 enabled.The session shape also resembles #24732: a tool-heavy, long-lived session and a roughly 43.5 MB rollout, although this case has fewer tool calls and failed at a lower visible token count.
I can provide a sanitized event timeline or a session/feedback ID if maintainers need it.
diegomrsantos commentedon Sep 14, 2026
Confirmed with Codex Desktop
26.908.40834, bundled CLI0.154.0-alpha.6.2, and source atb6a5d5bb14d6c3d6a656ca8d2274d6d4259ac57d.I put an adaptation of the proposed fix and four regression tests in diegomrsantos/codex#1. All four tests fail against the original behavior and pass with the fix.
Repeated stalls lasting several minutes disrupt my daily work and sometimes force me to interrupt tasks. This defect could contribute when an affected request uses SSE, but I have not captured this specific failure sequence during a live incident.
@jif-oai, could you triage this or route it to the current owner? I identified you through the recent changes to this SSE polling loop in #43408.
1320800521 commentedon Sep 20, 2026
Thanks — the four regression tests in your adaptation close an important gap between the deterministic loopback reproduction and a candidate source-level fix.
From the XBSTACK side, the useful release-level verification boundary is still the same pair:
response.failed+ immediate EOF should preserve the original terminal error.response.failed+ socket kept open should terminate from the terminal event rather than later surfacing only the SSE idle-timeout error.I’ll rerun the independent fixture against the first tagged Codex release that contains the fix (or against a maintainer-nominated commit if needed) and report the before/after result here.
fix(core): stop retrying terminal inference failures
jaysonsantos commentedon Sep 22, 2026
I have a matching production incident with Codex CLI
0.153.4against a custom OpenAI-compatible Responses endpoint.Observed evidence:
stream disconnected before completion: idle timeout waiting for SSE.I did not capture the original terminal
response.failedevent. Therefore, I cannot prove that this client defect masked the upstream error in this incident. The timing and final message match the documented failure shape.This incident also shows the recovery impact. The writer completed substantial local work, but the generic terminal error prevented automatic continuation. The caller could not distinguish an upstream model failure from a transport or infrastructure failure.
I can provide a sanitized event sequence if maintainers need more detail.
1320800521 commentedon Sep 23, 2026
Thanks — this is useful production evidence, and I would keep one scope boundary explicit.
Your incident strengthens the operational impact case: substantial tool work completed, the client ended with the generic SSE idle-timeout error, and a separate request through the same route succeeded shortly afterward. That makes a persistent route outage less likely.
It still does not prove that this exact incident contained a terminal
response.failedthat Codex then masked. I would separate the evidence into two classes:response.failedarrives, the socket stays open, and Codex waits until the idle timeout, replacing the original terminal error.If maintainers want to distinguish the two in production, the most useful next artifact would be a sanitized SSE/app-server event sequence immediately before the timeout. If it shows
response.failedwas already received and the client still waited, that would directly connect the production incident to this bug. If no terminal event arrived, it may be a separate upstream/transport stall with the same final symptom.From the XBSTACK side I’ll keep the release-level regression check unchanged: after a fix lands, both
response.failed + EOFandresponse.failed + held-open socketshould terminate with the original failure rather than a later idle-timeout replacement.fix(core): stop retrying terminal inference failures
fix(core): stop retrying terminal inference failures
fix(core): stop retrying terminal inference failures
fix(core): stop retrying terminal inference failures
mrstepanrf-a11y commentedon Sep 26, 2026
Additional production evidence + current-main check from 2026-09-26.
A Windows Codex Desktop incident (26.924.1866.0, bundled codex-cli 0.158.0-alpha.2) hit remote auto-compaction at 244,963 / 244,800 scope tokens. After two WebSocket compaction failures (
IO error: cannot decrypt peer's message) the client fell back to HTTP./responsesreturned 200 OK, then ~302.356 seconds later the runtime surfacedstream disconnected before completion: idle timeout waiting for SSE.I also inspected current
openai/codexmain today. Incodex-rs/codex-api/src/sse/responses.rs,response.failedstill becomes an error fromprocess_responses_event, butprocess_sse_with_treatmentstores that asresponse_error = Some(...)and continues reading; the timeout branch still returnsidle timeout waiting for SSEdirectly. So the terminal-failure/held-open-socket behavior described in this issue still appears present in current main.There is a tested fork patch in diegomrsantos#1 that treats
response.failedas terminal and returns immediately, with regression tests. I have not treated that fork as an official released fix.For the production incident, I do not have proof that a
response.failedevent arrived before the timeout, so this is compatible evidence, not causal proof. OpenAI request IDs are available in #45472 for backend correlation.fix(core): stop retrying terminal inference failures
fix(core): stop retrying terminal inference failures
fix(core): stop retrying terminal inference failures
fix(core): stop retrying terminal inference failures