Skip to content

Repeated image-context uploads cause network-wide latency spikes; throttling slows Codex steps #45109

Open

Description

@Kanturna

Repeated image-context uploads cause connection-wide latency spikes; throttling trades gaming latency for slower Codex steps

Summary

On Windows, an image-assisted Codex task repeatedly sent 11–16.5 MB compressed
requests. Process-attributed Windows network tracing and simultaneous pings
show that these uploads coincided with internet RTT rising from about 9 ms to
105–137 ms, plus occasional 900-ms timeouts. The local Ethernet gateway stayed
below 1 ms. Valorant was affected while Codex worked in the background.

A temporary 10-Mbit/s upload limit substantially improved RTT during a real
15-MB Codex upload, but increased the time spent transferring each request.
The user subsequently reported very slow steps in two tasks, removed the
limit, and reported that the network spikes returned. Only one of those two
tasks was identified and inspected; the other was not independently diagnosed.

Please investigate repeated transmission of accumulated tool images and
provide a supported way to bound serialized image-context size while retaining
useful task state. Bandwidth throttling alone is an unsatisfactory tradeoff.

Environment

  • Windows, wired gigabit Ethernet; actual WAN capacity was not independently measured.
  • Official Codex desktop package 26.908.4834.0.
  • Bundled CLI embedded version marker: 0.154.0-alpha.6.2; active executables
    matched the official package mirror. No custom binary was active.
  • Affected task: gpt-6-astra, reasoning effort ultra, ChatGPT authentication.
  • Actual measured requests used responses_http.
  • Request compression and compaction_image_budget were enabled.
  • Non-default setting: model_auto_compact_token_limit_scope = "body_after_prefix".
  • Tests performed September 12, 2026. Times below are UTC; MB means decimal MB.

Measured uploads and image-context growth

Application logs recorded:

Request time JSON bytes before zstd Bytes after zstd
19:14:43 15,127,472 11,141,163
19:15:02 16,795,345 12,391,935
19:15:29 20,210,291 14,956,043

The three compressed bodies total 38,489,141 bytes. Independently, ETW
attributed about 38.62 MB sent during the one-minute window to codex.exe,
96.90% of captured TCP/UDP send bytes; Valorant accounted for 1.203 MB.

Structural inspection of the affected history through 19:15:35 found 13 inline
image occurrences totaling 19,567,858 data-URL characters after the last
replacement history at 18:58:26. All came from new custom_tool_call_output
items after that compaction. This closely explains the roughly 20.2-MB JSON
request, but is not a captured/decrypted HTTP body or exact image deduplication
analysis. The evidence does not establish that all those images were obsolete.

Runtime logs showed about 99,854–106,271 tokens relevant to auto-compaction,
against a 244,800 threshold. Substantial image-byte growth therefore occurred
without reaching the configured token threshold. zstd was already working;
enabling compression again would not fix the issue.

Network mitigation experiment

We compared two approximately 60-second windows, using parallel pings to the
gateway, 1.1.1.1 and 8.8.8.8, adapter byte counters, and the
Microsoft-Windows-Kernel-Network ETW provider. Attribution used the event
payload PID for TCP/UDP send events 10/26/42/58, not generic process I/O counters
or the kernel execution PID. No speed-test traffic was generated.

The intervention was a temporary Windows QoS rule matching codex.exe, TCP
destination port 443, at 10,000,000 bit/s.

Observation Before With limit
Ping samples 82 84
Successful Cloudflare RTTs 9–137 ms 9–14 ms
Mean successful Cloudflare RTT while adapter upload exceeded 0.5 MB/s 86.80 ms, 10 samples 10.41 ms, 17 samples
Samples where both public targets timed out at 900 ms 3 1
Peak total adapter upload 40.28 Mbit/s 10.57 Mbit/s

The after-window included a newly logged 15,018,283-byte compressed request and
15,063,151 Codex-attributed send bytes. It was not an idle-network comparison.
Gateway replies remained below 1 ms in both windows. Failed pings were excluded
from RTT statistics, rather than interpreted as their recorded zero-ms value.

This strongly supports upload-induced WAN queueing, but router queues were not
directly instrumented. The windows contain different request counts and only
one minute each; they do not establish a reliable packet-loss rate or complete
elimination of disruptions.

Subsequent slowdown and rollback

A later image-assisted task still sent about 16–16.5 MB per request. Several
logged request-start-to-HTTP-200 intervals were about 15 seconds. That interval
includes upload and server time; it is not a pure upload measurement or a
controlled estimate of the limit's slowdown. At 10 Mbit/s, transferring 16.5 MB
alone takes approximately 13.2 seconds in the ideal single-flow case.

That task also logged incremental request failed, websocket reuse properties didn't match, followed later by falling back to HTTP. Successful HTTP
responses and subsequent sampling continued, so this was not evidence of a
permanent hang. The cause of the WebSocket failure was not established.

The user confirmed running the rollback and reported renewed ping spikes.
The rollback/recurrent-spike report was not followed by a fresh ETW comparison.
Manual /compact was suggested but could not be tried at that time; no
post-compaction improvement is claimed. No task history was edited.

Requested improvements

  1. Bound aggregate image-context bytes independently of token-based compaction,
    with safe summarization/eviction of old tool images and preservation of
    references needed for the task.
  2. Investigate repeated full image-history uploads on the HTTP fallback path;
    expose when incremental reuse is unavailable and why.
  3. Show request size and distinguish uploading, awaiting server response, and
    tool execution so long steps are understandable.
  4. Offer upload pacing or bandwidth controls as an optional mitigation, without
    treating them as a substitute for reducing redundant context transfer.

Related: #43015 describes large
image-history requests despite zstd. This report adds independent
process-attributed network measurements and a measured throttling tradeoff.
It does not assume the two cases have an identical internal root cause.

This report contains aggregate measurements and selected non-secret log fields.
Private prompts, images, task IDs, local paths, authentication data, raw traces,
and the full conversation are deliberately not attached.

Activity

  1. added
    bugSomething isn't working
    appIssues related to the Codex desktop app
    windows-osIssues related to Codex on Windows systems
    contextIssues related to context management (including compaction)
    connectivityIssues involving networking or endpoint connectivity problems (disconnections)
    on Sep 12, 2026
  2. github-actions commented on Sep 12, 2026

    @github-actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  3. oyishyi commented on Sep 20, 2026

    @oyishyi

    I'm seeing a similar problem in Codex Desktop on Windows x64. Adding measurements from my affected conversation (also reported in #46761).

    The main user-visible impact is extremely slow progress between agent loops. Successive model calls continue to send tens of MiB of historical image data during ordinary coding work, making the conversation painfully slow even when the current step does not require inspecting the old images.

    Debug findings from September 18, 2026:

    Measurement Result
    Inline user images in an inspected history snapshot 30
    Combined image data-URL size 39,408,084 bytes / 37.58 MiB
    Image data as a share of serialized history Approximately 99.6%
    Requests logged between 22:30 and 23:00 UTC+8 64
    Sum of compressed request-body sizes in that window 2,780.74 MiB / 2.716 GiB
    Individual compressed request-body sizes Approximately 49.9 MiB, peaking at 78.34 MiB; other requests were still around 27.6–28.6 MiB
    • Images were stored inline as data:image/png;base64,.... Per-image SHA-256 comparison against an earlier history snapshot showed that the same 30 image payloads were retained unchanged.
    • Request logs identified transport=responses_http, method=POST, targeting /backend-api/codex/responses.
    • Reading 12 additional screenshots through view_image added approximately 39.6 MB of image output, coinciding with an increase in request sizes.
    • In the scoped task logs, I found no evidence that network retries were the main source of this volume.
    • The affected bundled runtime reported 0.155.0-alpha.9.

    These are local history and request-log measurements, not a packet capture. The 99.6% figure describes the inspected history snapshot, not every complete HTTP request. I have not separately timed upload versus server processing, so I cannot give a precise latency breakdown or establish that this has exactly the same root cause as the original report. The repeated large payloads are consistent with the severe per-loop delays I experience.

    Reusing images through stable references or attachment IDs, loading older images on demand, and allowing users to remove old image payloads from active context would help address this. Exposing request size and upload time separately from model processing would also make the delays much easier to diagnose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    appIssues related to the Codex desktop appbugSomething isn't workingconnectivityIssues involving networking or endpoint connectivity problems (disconnections)contextIssues related to context management (including compaction)performancewindows-osIssues related to Codex on Windows systems

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions