Repository navigation
Repeated image-context uploads cause network-wide latency spikes; throttling slows Codex steps #45109
Description
Repeated image-context uploads cause connection-wide latency spikes; throttling trades gaming latency for slower Codex steps
Summary
On Windows, an image-assisted Codex task repeatedly sent 11–16.5 MB compressed
requests. Process-attributed Windows network tracing and simultaneous pings
show that these uploads coincided with internet RTT rising from about 9 ms to
105–137 ms, plus occasional 900-ms timeouts. The local Ethernet gateway stayed
below 1 ms. Valorant was affected while Codex worked in the background.
A temporary 10-Mbit/s upload limit substantially improved RTT during a real
15-MB Codex upload, but increased the time spent transferring each request.
The user subsequently reported very slow steps in two tasks, removed the
limit, and reported that the network spikes returned. Only one of those two
tasks was identified and inspected; the other was not independently diagnosed.
Please investigate repeated transmission of accumulated tool images and
provide a supported way to bound serialized image-context size while retaining
useful task state. Bandwidth throttling alone is an unsatisfactory tradeoff.
Environment
- Windows, wired gigabit Ethernet; actual WAN capacity was not independently measured.
- Official Codex desktop package
26.908.4834.0. - Bundled CLI embedded version marker:
0.154.0-alpha.6.2; active executables
matched the official package mirror. No custom binary was active. - Affected task:
gpt-6-astra, reasoning effortultra, ChatGPT authentication. - Actual measured requests used
responses_http. - Request compression and
compaction_image_budgetwere enabled. - Non-default setting:
model_auto_compact_token_limit_scope = "body_after_prefix". - Tests performed September 12, 2026. Times below are UTC; MB means decimal MB.
Measured uploads and image-context growth
Application logs recorded:
| Request time | JSON bytes before zstd | Bytes after zstd |
|---|---|---|
| 19:14:43 | 15,127,472 | 11,141,163 |
| 19:15:02 | 16,795,345 | 12,391,935 |
| 19:15:29 | 20,210,291 | 14,956,043 |
The three compressed bodies total 38,489,141 bytes. Independently, ETW
attributed about 38.62 MB sent during the one-minute window to codex.exe,
96.90% of captured TCP/UDP send bytes; Valorant accounted for 1.203 MB.
Structural inspection of the affected history through 19:15:35 found 13 inline
image occurrences totaling 19,567,858 data-URL characters after the last
replacement history at 18:58:26. All came from new custom_tool_call_output
items after that compaction. This closely explains the roughly 20.2-MB JSON
request, but is not a captured/decrypted HTTP body or exact image deduplication
analysis. The evidence does not establish that all those images were obsolete.
Runtime logs showed about 99,854–106,271 tokens relevant to auto-compaction,
against a 244,800 threshold. Substantial image-byte growth therefore occurred
without reaching the configured token threshold. zstd was already working;
enabling compression again would not fix the issue.
Network mitigation experiment
We compared two approximately 60-second windows, using parallel pings to the
gateway, 1.1.1.1 and 8.8.8.8, adapter byte counters, and the
Microsoft-Windows-Kernel-Network ETW provider. Attribution used the event
payload PID for TCP/UDP send events 10/26/42/58, not generic process I/O counters
or the kernel execution PID. No speed-test traffic was generated.
The intervention was a temporary Windows QoS rule matching codex.exe, TCP
destination port 443, at 10,000,000 bit/s.
| Observation | Before | With limit |
|---|---|---|
| Ping samples | 82 | 84 |
| Successful Cloudflare RTTs | 9–137 ms | 9–14 ms |
| Mean successful Cloudflare RTT while adapter upload exceeded 0.5 MB/s | 86.80 ms, 10 samples | 10.41 ms, 17 samples |
| Samples where both public targets timed out at 900 ms | 3 | 1 |
| Peak total adapter upload | 40.28 Mbit/s | 10.57 Mbit/s |
The after-window included a newly logged 15,018,283-byte compressed request and
15,063,151 Codex-attributed send bytes. It was not an idle-network comparison.
Gateway replies remained below 1 ms in both windows. Failed pings were excluded
from RTT statistics, rather than interpreted as their recorded zero-ms value.
This strongly supports upload-induced WAN queueing, but router queues were not
directly instrumented. The windows contain different request counts and only
one minute each; they do not establish a reliable packet-loss rate or complete
elimination of disruptions.
Subsequent slowdown and rollback
A later image-assisted task still sent about 16–16.5 MB per request. Several
logged request-start-to-HTTP-200 intervals were about 15 seconds. That interval
includes upload and server time; it is not a pure upload measurement or a
controlled estimate of the limit's slowdown. At 10 Mbit/s, transferring 16.5 MB
alone takes approximately 13.2 seconds in the ideal single-flow case.
That task also logged incremental request failed, websocket reuse properties didn't match, followed later by falling back to HTTP. Successful HTTP
responses and subsequent sampling continued, so this was not evidence of a
permanent hang. The cause of the WebSocket failure was not established.
The user confirmed running the rollback and reported renewed ping spikes.
The rollback/recurrent-spike report was not followed by a fresh ETW comparison.
Manual /compact was suggested but could not be tried at that time; no
post-compaction improvement is claimed. No task history was edited.
Requested improvements
- Bound aggregate image-context bytes independently of token-based compaction,
with safe summarization/eviction of old tool images and preservation of
references needed for the task. - Investigate repeated full image-history uploads on the HTTP fallback path;
expose when incremental reuse is unavailable and why. - Show request size and distinguish uploading, awaiting server response, and
tool execution so long steps are understandable. - Offer upload pacing or bandwidth controls as an optional mitigation, without
treating them as a substitute for reducing redundant context transfer.
Related: #43015 describes large
image-history requests despite zstd. This report adds independent
process-attributed network measurements and a measured throttling tradeoff.
It does not assume the two cases have an identical internal root cause.
This report contains aggregate measurements and selected non-secret log fields.
Private prompts, images, task IDs, local paths, authentication data, raw traces,
and the full conversation are deliberately not attached.
Activity
github-actions commentedon Sep 12, 2026
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
oyishyi commentedon Sep 20, 2026
I'm seeing a similar problem in Codex Desktop on Windows x64. Adding measurements from my affected conversation (also reported in #46761).
The main user-visible impact is extremely slow progress between agent loops. Successive model calls continue to send tens of MiB of historical image data during ordinary coding work, making the conversation painfully slow even when the current step does not require inspecting the old images.
Debug findings from September 18, 2026:
data:image/png;base64,.... Per-image SHA-256 comparison against an earlier history snapshot showed that the same 30 image payloads were retained unchanged.transport=responses_http,method=POST, targeting/backend-api/codex/responses.view_imageadded approximately 39.6 MB of image output, coinciding with an increase in request sizes.These are local history and request-log measurements, not a packet capture. The 99.6% figure describes the inspected history snapshot, not every complete HTTP request. I have not separately timed upload versus server processing, so I cannot give a precise latency breakdown or establish that this has exactly the same root cause as the original report. The repeated large payloads are consistent with the severe per-loop delays I experience.
Reusing images through stable references or attachment IDs, loading older images on demand, and allowing users to remove old image payloads from active context would help address this. Exposing request size and upload time separately from model processing would also make the delays much easier to diagnose.