Skip to content

Remote SSH disconnects when large plugin/list response blocks WebSocket pongs #41573

Open

Description

@lingyang-kong

What version of the Codex App are you using (From “About Codex” dialog)?

26.825.41651

What subscription do you have?

ChatGPT Pro Lite

What platform is your computer?

Client: Darwin 24.6.0 arm64 arm; server: Linux 6.8.0-111-generic x86_64 x86_64

What issue are you seeing?

Codex Desktop Remote SSH repeatedly disconnects with WebSocket code 1006, although SSH remains available and the persistent remote app-server stays alive.

A protocol probe identified this sequence:

  1. plugin/list returns approximately 7.57 MB containing over 3,300 remote plugin summaries.
  2. The response is emitted as one unfragmented WebSocket frame.
  3. While that frame drains through the SSH proxy, pong frames cannot interleave.
  4. Desktop continues sending pings, but its 60-second no-pong watchdog eventually terminates the healthy proxy.

The app-server does not appear globally stalled:

Cached plugin/list:          0.31 s,  7,568,723 bytes, 1 frame
Forced plugin/list:           9.47 s,  7,573,180 bytes, 1 frame
Concurrent config/read: 2-6 ms

During an actual failure, server-side monitoring showed continuing proxy output, acknowledged TCP traffic, and an empty TCP send queue. The persistent app-server retained the same PID after the proxy was terminated.

A controlled slow-reader test reproduced the behavior without SSH: once the large frame began, subsequent pongs and small responses remained unavailable until the frame completed, causing a no-pong watchdog to fire.

What steps can reproduce the bug?

  1. Enable plugins and remote_plugin on a Linux Codex host.
  2. Connect using Codex Desktop Remote SSH.
  3. Trigger or wait for a background plugin/list.
  4. Observe the request timeout, WebSocket close code 1006, and reconnection.
  5. Confirm that SSH and the persistent app-server remain alive.

The failure also reproduced with SSH multiplexing disabled and a fresh authenticated SSH connection.

What is the expected behavior?

Large app-server responses should not block WebSocket control traffic or disconnect the remote host.

Additional information

As a size-control experiment, setting

[features]
remote_plugin = false

reduced the response from approximately 7.57 MB to 319 KB while retaining bundled and locally curated plugins.

Related: #37401, #34244, #36518.

Activity

  1. added
    bugSomething isn't working
    app-serverIssues involving app server protocol or interfaces
    connectivityIssues involving networking or endpoint connectivity problems (disconnections)
    skillsIssues related to skills
    on Aug 29, 2026
  2. TheNewMikeMusic commented on Sep 5, 2026

    @TheNewMikeMusic

    Possible related Android Remote / hosted relay case with fresh payload measurements. This is not confirmation of the SSH-specific unfragmented-frame / no-pong failure described above.

    Environment:

    • Official macOS Desktop 26.901.41600, build 7982
    • Official bundled CLI/app-server 0.153.4
    • macOS 26.6.2, arm64
    • Android Remote over cellular data with a VPN; exact Android app version not collected

    The phone can open tasks, but refreshing/reconnecting Remote remains sluggish after restarting the host app. Actual host logs contain initialization requests from app_server.client_name="codex_chatgpt_android_remote" to plugin/list and mcpServerStatus/list.

    Read-only, isolated replays against the official bundled app-server produced:

    Request Local elapsed time Response size
    plugin/list 4.5–4.8 s across two runs 8,366,624 bytes, actual JSON-RPC response
    plugin/installed control 47.1 ms 119,848 bytes, actual JSON-RPC response
    mcpServerStatus/list 7.9 s, one run About 2.77 MB when reserialized by the diagnostic client
    thread/list for pins Under 10 ms 2 or 5 tasks, depending on provider filter

    The full remote marketplace contained 3,710 plugin summaries, of which 33 were marked installed. The remote marketplace accounted for 8,337,629 bytes of compact UTF-8 JSON; its interface fields alone accounted for about 6.31 MB.

    Replay details:

    • An auxiliary official codex app-server --listen stdio:// was used, with CODEX_INTERNAL_APP_SERVER_REMOTE_CONTROL_DISABLED=1 to avoid registering an additional Remote host.
    • Initialize with capabilities.experimentalApi: true.
    • Compare plugin/list and plugin/installed using the same cwds, with default marketplace options and without forcing a catalog refetch.
    • Measure time to the matching RPC response and the byte length of its UTF-8 line; do not retain the catalog payload.

    Limits:

    • Android's exact request parameters, its spinner's critical path, WebSocket frame fragmentation, and no-pong termination were not captured.
    • These are isolated local RPC measurements, not measured phone end-to-end connection times; the request durations should not be summed as one connection duration.
    • The VPN path remains a possible contributor. Four unauthenticated Mac HTTPS reachability probes took 0.72–0.80 s, but that is not an authenticated Remote health test.
    • We have not disabled remote plugins as a workaround, to preserve the user's installed plugin behavior.

    In addition to transport fragmentation / control-frame scheduling, could the mobile initialization path avoid fetching the full marketplace before ready state? If it only needs installed plugin mentions, plugin/installed was about 70 times smaller in this environment. Full catalog loading could remain on the explicit marketplace browsing path.

    No private task content, task/host/account IDs, credentials, catalog payload, or raw logs are included.

  3. rellaqaq commented on Sep 7, 2026

    @rellaqaq

    Additional independent Android/hosted-Remote measurements, plus a caveat about the remote_plugin=false workaround. This does not confirm the SSH-specific no-pong failure for hosted Remote.

    Environment: official macOS Desktop 26.901.51231 / build 8109, bundled app-server 0.153.4. One Android phone uses a WireGuard tunnel over cellular data. The owner has now supplied the About screenshot: ChatGPT Android 1.2026.244 (21). Reported symptom is intermittent searching for the desktop or reconnecting, not initial pairing failure.

    Two isolated replays of the official bundled stdio app-server, using CODEX_INTERNAL_APP_SERVER_REMOTE_CONTROL_DISABLED=1, experimentalApi: true, identical cwds, omitted marketplace options, and no forced refetch:

    Request / process-only configuration UTF-8 JSON-RPC response bytes Rows Installed / enabled
    Default plugin/list 7,792,089 3,478 19 / 19
    Default plugin/installed 42,168 22 22 / 22
    remote_plugin=false, plugin/list 344,845 196 15 / 15
    remote_plugin=false, plugin/installed 26,276 15 15 / 15

    Default plugin/list took 2,327 / 2,269 ms cold and 837 / 499 ms warm; plugin/installed took 13 / 12 ms. These are local individual RPC timings, not mobile end-to-end connection measurements. Different list/installed counts reflect the distinct responses, not an assumed common enumeration.

    Crucially, installed-only name comparison across the feature override found seven installed/enabled plugin names absent, with no replacement names. Thus, in this environment, disabling remote_plugin is not merely suppressing marketplace discovery while retaining every installed plugin. We did not persist that override. No catalog payloads were retained, and all auxiliary processes exited without registering another Remote owner.

    An installed-only initialization path is about 185 times smaller here while retaining all 22 entries. Could the official Android ready-state path use plugin/installed when it needs installed mentions and defer the full catalog until marketplace browsing? Alternatively, is there a supported installed-preserving configuration for this case?

    Limits: Android's exact RPC parameters, spinner critical path, relay ACK occupancy, and control-frame scheduling were not captured. Earlier separate host-side TLS EOF reconnects recovered automatically in 1–4 seconds. A later outage lasted about70seconds and included a30-second required-server-token refresh timeout followed by a logged35-second retry deferral. These are separate transport/retry observations, not proof that the catalog caused the disconnects. Current hosted relay source segments envelopes, so the original SSH unfragmented-frame explanation cannot simply be transferred. No confirmed production fix is claimed.

    Owner update,2026-09-07: a fresh screenshot shows the Mac indicator green while a conversation remains on loading skeletons. The owner reports that manually disconnecting/reconnecting mobile Remote was necessary to restore loading. Host transport had reconnected before that screenshot, and Android thread/resume requests reached the host around the same period. This establishes the recurrent user-visible failure, but not whether an undelivered response, client recovery state or catalog initialization was responsible. No configuration change, plugin removal or pairing reset was performed.

    Only aggregate diagnostics are included; no account/task/host identifiers, private endpoints, credentials, plugin payloads, or raw logs.

  4. bradleygibsongit commented on Sep 10, 2026

    @bradleygibsongit

    Independent macOS → Linux Remote SSH reproduction, tested September 9–10, 2026. Upgrading the remote CLI/app-server from 0.153.4 to 0.154.0 did not resolve the catalog timeout. Setting features.remote_plugin = false and restarting the remote app-server restored responsiveness in our tests and in actual desktop use.

    Environment:

    • macOS desktop app 26.903.61454, build 8378.
    • Linux x86_64 remote host, official standalone Codex installation.
    • Confirmed running app-server 0.154.0, both through the version command and desktop logs.
    • SSH over IPv6; SSH compression off for the before/after comparison.

    Before the feature change:

    • Desktop operations against the remote host were broadly unresponsive.
    • Logs during the investigation showed repeated WebSocket 1006 disconnects and 30-second plugin/list and thread/start timeouts.
    • On 0.154.0, plugin/list returned 8,178,596 bytes, including 3,642 available remote plugin summaries (not installed-plugin count).
    • The response completed through the server's local Unix socket in 1.042 s, but exceeded our 25-second response deadline through SSH.
    • Small SSH requests in that run remained fast: account/read 196 ms, model/list 198 ms.

    After setting the following in the remote configuration and restarting the app-server:

    [features]
    remote_plugin = false
    • plugin/list shrank to 118,865 bytes and completed over SSH in 3.602 s, then 2.061 s.
    • Subsequent account/read requests stayed around 190 ms, including checks at 20, 40, and 60 seconds on the same connection.
    • No new desktop timeout/reconnect events appeared during the roughly 100-second log observation after reconnect.
    • The user confirmed that opening and using remote tasks was responsive now.

    Measurement method: read-only JSON-RPC requests over the existing app-server's Unix socket or its supported codex app-server proxy SSH transport, initialized with experimentalApi: true; plugin/list parameters were {}, without forced refetch. Byte counts are the received JSON-RPC message payload sizes.

    Limits: the network also showed constrained throughput/retransmissions during earlier testing. We did not directly capture WebSocket fragmentation or prove the no-pong watchdog mechanism here. This is a short-term workaround result, not proof of long-term stability. No plugins were uninstalled, but we have not verified that disabling remote_plugin preserves every installed plugin's availability.

    This adds a reproduction on 0.154.0 and a measured workaround result supporting investigation of full-catalog traffic on the interactive SSH connection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    appIssues related to the Codex desktop appapp-serverIssues involving app server protocol or interfacesbugSomething isn't workingconnectivityIssues involving networking or endpoint connectivity problems (disconnections)remoteskillsIssues related to skills

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions