Repository navigation
thread/resume is effectively quadratic on large active threads, blocking Remote steering #38787
Description
Summary
Opening or steering a very large active Codex thread from iOS Remote can time out even though pairing, authentication, transport, and the host app-server are healthy.
The same failure occurs through a client that sends a follow-up to an existing thread: the client first performs thread/resume, app-server parses the rollout successfully, then remains CPU-bound while composing the resume response. The follow-up is never submitted.
For the affected thread, using the already-loaded Codex Desktop composer bypassed thread/resume and delivered the steer immediately.
Environment
- Codex CLI / app-server:
0.147.0 - Codex Desktop:
26.810.50856 - iOS Remote client:
1.2026.209 - Host: Linux x86_64 (
6.17.0-14-generic) - Rollout size when reproduced: 22,617 records, dominated by one long active turn with many tool and subagent items
Observed behavior
Each Remote attempt completed rollout parsing quickly:
Resumed rollout with 22617 items, parse errors: 0
It then stopped making progress at:
composing running thread resume response ... active_turn_present=true ... active_turn_status=InProgress
During the stall:
- app-server consumed approximately one CPU core continuously;
- RSS was approximately 1.1 GB;
- the client eventually surfaced a generic failure;
- the user prompt was absent from the target rollout, confirming that steering never began;
- repeated iOS retries opened additional connections that all stalled at the same reconstruction point.
There were no concurrent 401, 403, duplicate-owner 409, pairing, or websocket-connectivity failures.
Reproduction
- Create or retain an active thread with a very large rollout, particularly one long turn containing thousands of tool/subagent items.
- Connect to the host through iOS Remote.
- Open that thread or send a prompt while its turn is active.
- Observe that the client initiates
thread/resumewith turn history included. - Rollout parsing completes, but the resume response does not return in practical time and the steer is never submitted.
- Open the same already-loaded thread in Codex Desktop and use its composer/Steer control; the message is recorded and handled immediately.
The failure also reproduced through the Codex thread-management follow-up path because it performs the same full thread/resume before steering.
Root cause
In app-server/src/request_processors/thread_lifecycle.rs, the loaded-thread resume path calls populate_thread_turns_from_history whenever pending.include_turns is true:
That builds legacy API turns from every rollout item. During reconstruction, upsert_turn_item searches the growing item vector linearly for every incoming item:
if let Some(existing_item_index) = items
.iter()
.position(|existing_item| existing_item.id() == item.id())For a turn with thousands of items, repeated linear scans make reconstruction effectively quadratic. The stall begins after parsing and exactly where the logs say app-server is composing the running-thread response. The process behavior is consistent with CPU-bound reconstruction rather than a dead connection or authentication failure.
The same linear upsert was still present on main when checked on 2026-08-15.
Expected behavior
A Remote client should be able to open and steer a known active thread without material latency proportional to the square of its retained turn-item count. A failure should also be bounded and retryable rather than accumulating more expensive resume jobs.
Suggested fixes
Two complementary changes would address this:
- Metadata-only steering path: when the client already knows the thread and only needs its active turn ID, call
thread/resumewithexcludeTurns: true, then submitturn/steerwithexpectedTurnId. Do not serialize full history merely to steer an active thread. - Linear history reconstruction: maintain an item-ID-to-index map while building a turn, or otherwise avoid
items.iter().position(...)for every item. Pagination/bounds would provide additional protection for unusually large histories.
It would also help to coalesce or reject duplicate in-flight resumes for the same thread so repeated mobile retries do not amplify the CPU queue.
Workaround
Use the already-loaded Codex Desktop composer and explicitly select Steer. This bypassed full history reconstruction and delivered the prompt immediately. Avoid repeated Remote retries on the affected thread.
Privacy note
This report intentionally omits the thread ID and full rollout because the rollout contains private project conversation and tool output. The two sanitized log lines above identify the failing stage without including that content. I can provide narrower diagnostics if maintainers specify what is needed.
Related but not exact duplicates
- thread/resume hangs forever when codex thread is stuck "active" with no running turn (0.146.0) #37047:
thread/resumehangs on a stale active listener with no running turn. Here the turn is genuinely in progress and app-server is CPU-bound reconstructing 22k+ history items. - Codex Mobile remote prompt stuck in Status Sending after steer blocks further input #36831: mobile prompt remains stuck in
Status Sending; adjacent symptom, but this report identifies the preceding resume-reconstruction bottleneck. - Android Remote Control times out listing WSL threads when thread/list scans rollouts #36416: Remote
thread/listscans many rollout files. Similar principle—Remote blocking on unnecessary history work—but a different method and data structure.
Activity
mrcoinzzz commentedon Aug 16, 2026
I can reproduce what looks like a related large-thread Remote failure, with a particularly strong iPad/iPhone comparison.
Environment:
Observed behavior on iPad:
Important A/B control:
This makes the failure appear strongly correlated with individual thread length/history/state rather than the project, Mac host, network, or Remote pairing itself.
It may be the same underlying large-thread resume/history reconstruction problem described in this issue, but on iPad the symptom is severe enough to freeze the entire ChatGPT app and then terminate it on foreground restore.
Happy to provide iPad model, iPadOS version, ChatGPT app version, or any diagnostics/crash logs if useful.
mrcoinzzz commentedon Aug 17, 2026
Follow-up after testing with a fresh thread this morning:
Thread size is clearly a strong aggravating factor, but it does not appear to be the sole trigger.
So creating a fresh thread greatly improves reliability on the iPad, but does not eliminate the underlying Remote freeze. The fact that very large threads can trigger the freeze before their contents render may be useful diagnostically - it suggests the large-thread failure can occur during loading/hydration/resume rather than only while rendering an already-loaded conversation.
This may therefore be an underlying iPad Remote instability that is significantly amplified by large thread history, rather than thread size being the only cause.
mrcoinzzz commentedon Aug 18, 2026
Practical workaround update, in case it helps other iPad Remote users while this is being investigated:
I have now changed my long-running Codex workflows to use fresh threads regularly. For my daily helpdesk workflow, I start a new thread each morning rather than continuing the very large historical thread.
To preserve continuity, the new thread receives only a small handover/bootstrap prompt. That prompt tells Codex to read a small project file, which in turn points it to the other persistent instructions/resources it needs. Codex can then reconstruct the working context from the project itself rather than requiring the previous conversation history.
Result on my iPad:
For workflows where the durable state already lives in project files, a ticket system, repository, etc., this seems to be a useful temporary workaround: keep Remote threads relatively short and bootstrap each fresh thread from persistent project resources rather than carrying a huge conversation indefinitely.
dbjaredh commentedon Sep 4, 2026
Adding a current reproduction that appears consistent with this root cause and may help establish user impact beyond the original environment.
Observed 2026-09-03/04 using Codex Desktop with ChatGPT iOS Remote:
Recovery attempts that did NOT fix the affected thread:
This is particularly consistent with the distinction described in this issue: the already-loaded Desktop thread can continue without paying the full
thread/resumereconstruction cost, while Remote must resume/hydrate the thread and can stall on a sufficiently large history.I cannot provide the private thread contents publicly, but the failure is persistent and reproducible on this one thread. If maintainers want targeted diagnostics, useful next data to request would be: Codex Desktop build, ChatGPT iOS build, host OS, sanitized thread/rollout item counts, timestamps for a failed Remote open/resume, and app-server log lines surrounding
thread/resume/composing running thread resume response.One UX/reliability improvement in addition to the algorithmic fix: if Remote hydration exceeds a bounded threshold, fall back to metadata-only/partial history and allow the user to steer/monitor the active turn rather than leaving the mobile representation permanently stale. Incremental/paginated hydration plus an explicit retry/resume-from-cursor path would also prevent one oversized thread from becoming effectively unusable on mobile.
Related reproduction was also added to #28480.
skyzer commentedon Sep 7, 2026
Adding evidence from 2026-09-07 on a newer release. I can use a small session through iOS Remote, but cannot open existing large sessions. This case involves an already-loaded idle thread, rather than an in-progress turn.
Environment
codex_chatgpt_ios_remote, version1.2026.237Host evidence
For the same iOS
thread/resumerequest, on September 7 (UTC):Approximately 19 seconds is spent reading/parsing before response composition begins. Existing session content remains readable through desktop thread-read tooling. Another iOS-requested session has 225,684 records and an approximately 13 GiB rollout, but I do not have a comparable completed timing for it.
Source corroboration in 0.153.4
include_turns = !params.exclude_turns; for legacy threads, either included turns orinitialTurnsPagetriggers persisted history loading.The request parameters were not captured, so this does not distinguish full-history inclusion from
initialTurnsPage. I have not profiled CPU or established that the original issue's quadratic behavior is the precise bottleneck here. The final failure stage after parsing—construction, transfer, or iOS rendering—remains unmeasured.A metadata-only attachment followed by progressive history loading would let me continue the same existing task. For legacy sessions, paging should also avoid rereading the entire rollout. Useful diagnostics would separate parse/projection/serialization times, response bytes, and iOS decode/render time, plus capture the client's
excludeTurns/initialTurnsPagesettings.No private session contents, thread/request IDs, credentials, or local paths are included. Session files have not been modified or truncated.