[macOS Desktop 26.1007, build 20156] model/list expires after 30s in queue; Local Work startup repeatedly times out on plugin/installed #53197

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   appIssues related to the Codex desktop appIssues related to the Codex desktop app

   app-serverIssues involving app server protocol or interfacesIssues involving app server protocol or interfaces

   skillsIssues related to skillsIssues related to skills

   on Oct 11, 2026
2. github-actions commented on Oct 11, 2026

   

   github-actionsbot

   on Oct 11, 2026 – with GitHub Actions

   Contributor

   More actions

   Potential duplicates detected. Please review them and close your issue if it is a duplicate.

   • [macOS Desktop 26.1002] Queued messages never send; stopping or forking chat is required, with submission timeout #52423

   Powered by Codex Action
3. 0xdevalias commented on Oct 17, 2026

   

   0xdevalias

   on Oct 17, 2026

   More actions

   Independent current-build macOS confirmation, with a distinction between queued expiry and dispatched-request abandonment.

   In the initial captured interval: 631 queue rejections, including 29 critical account/auth rejections. At 11:20–11:21 AEDT there were 243 distinct observed plugin/installed request IDs (121 responses + 122 failures), 49 account/read and 51 getAuthStatus IDs. Failures in the first burst involved ten renderer windows. These are requests, not an asserted fixed demand per window. At the original stuck prompt's 12:48 enqueue there was no queue rejection or transport closure, so these bursts do not establish that prompt's initiating cause.

   Actual host scheduler vR in src-DCNiwad_.js: total active 8; noncritical active <=5; background active <=3; queued caps background 128, critical 16, interactive 64. Eight is the TOTAL active limit, not eight slots reserved for critical work. Critical requests can use remaining total capacity. There are additional metadata/thread/catalog lane limits. plugin/installed is absent from hR's coalescing allowlist (plugin/list is present).

   Actual-code tests show 132 identical plugin-inventory requests produce 3 active + 128 queued + 1 rejection. Background fullness alone does not reject critical work: account/auth fan-out can additionally fill the remaining active slots and all 16 critical queued slots, after which critical requests are rejected. The real rejection snapshots reach these limits.

   Demand: shared asset XZr inventory query has 30,000-ms timeout, retry=false, staleTime=0; outer W2 UI/executor queries include consumer/root/host keys, with local staleTime one minute (hosted five minutes). StaleTime is eligibility for another fetch, not proof of a once-per-minute polling timer. Different renderers and keys multiply demand. Account/auth state uses paired getAccount/getAuthMethod calls and readiness/auth-status/RPC-recovery listeners; FGr/IGr deduplicate only in their renderer/client context. Turn preparation and other consumers also call these methods. refreshToken:false is used on the underlying reads, so this is not evidence that every read refreshes an expired token.

   Dispatched timeout retention: extracting and invoking the actual abandonDestination method with a dispatched ordinary window request leaves plugin/installed, account/read, and getAuthStatus in pendingRequests; it does not call scheduler completion. Positive controls: dispatched plugin/list is detached and thread/resume completes. A renderer timeout is therefore not necessarily host-slot cancellation. A late response or connection failAll/disposal is needed for ordinary dispatched reads to release their slot. This agrees with #50812; closing a window is not a guaranteed cancellation of dispatched host work.

   Of 151 renderer timeouts, 148 have no matched response_routed in the reviewed logs; that absence alone does not distinguish queued expiry from dispatched retention. Two matched plugin calls spent approximately 29.7–30.0 seconds queued and only 483/556 ms executing, exceeding their renderer deadline. No current live pending inventory was available to identify every retained request or prove that all eight remain poisoned now.

   Fix/diagnostics: host-level single-flight/cache for inventory and account/auth reads; admission/backpressure; consistent end-to-end deadlines; explicit cancellation or safe detached-read accounting; age/method/origin/connection-generation inventory for in-flight and queued requests; oldest age, depth/high-water, coalesced/rejected/expired counts and actionable degraded-state UI. These are Desktop main-process queues, separate from Rust app-server queue_1.sqlite. Hardcoded JS limits are not a supported user configuration. Enlarging caps alone can prolong stalls or increase backend load. No defensible maximum number of open windows follows from eight RPC slots; one busy window can fill a lane and retained renderers also contribute demand.

   Related: #50812 and #43525 (shared queue/control latency); #53677 (avoidable denied catalog reads, usually fast rather than continuously occupying slots here).

   Verified 2026-10-17: Codex Desktop macOS arm64 26.1007.71326, build 20398, macOS 26.6 (25G72). Installed archive Contents/Resources/app.asar SHA-256 2f231a64e6dd30111bcd09e03c149cd06c8679b25914842772423251c4608ded.

   Build-specific source provenance:

   • webview/assets/app-shared-8340f229c2b3.js: SHA-256 3b4cabc382e71f0da7398393a619d7c62d13dfae598a5fe2e9880ab8d31d1bf5.
   • .vite/build/main-CwxgTOle.js: SHA-256 87d52265eb13608a380fabcac47ecf13a39b981115c0960c18af563ea6d32d51.
   • .vite/build/src-DCNiwad_.js: SHA-256 86fb2066bbc2f7428864b76b46ef570b09eb1e92e81e9d6f089f5b048a66df25.

   Source observations below are version-specific. Offline probes evaluate extracted shipped methods in a Node VM with synthetic dependencies; they do not establish an unlogged live race. No app patch/restart, queue edit, direct send, or SQLite mutation was performed. Private text, annotations, account identifiers, task IDs, and raw logs are excluded.

   Build-specific offline runner: Python 3 + Node.js v26.8.2 used

   Save and run this Python script against the stated build. It reads the archive and uses a temporary directory only. A different build may change extraction boundaries.

   HARNESS = r'''
   'use strict';
   const fs = require('node:fs');
   const vm = require('node:vm');
   const assert = require('node:assert/strict');
   const acorn = require('internal/deps/acorn/acorn/dist/acorn');
   const path = require('node:path');
   const dir = __dirname;
   const shared = fs.readFileSync(path.join(dir, 'source/shared.js'), 'utf8');
   const main = fs.readFileSync(path.join(dir, 'source/main.js'), 'utf8');
   function expression(source, offset) {
     let n = acorn.parseExpressionAt(source, offset, {ecmaVersion: 'latest'});
     if (n.type === 'SequenceExpression') n = n.expressions[0];
     return source.slice(n.start, n.end);
   }
   function func(source, name) { return expression(source, source.indexOf('function ' + name + '(')); }
   const results = [];
   function test(name, action) { action(); results.push({name, pass: true}); }
   const retryGlobals = {Njn: 3, OW: class extends Error {}};
   retryGlobals.Mjn = vm.runInNewContext('(' + func(shared, 'Mjn') + ')', retryGlobals);
   retryGlobals.jjn = vm.runInNewContext('(' + func(shared, 'jjn') + ')', retryGlobals);
   const retry = vm.runInNewContext('(' + func(shared, 'Bq') + ')', retryGlobals);
   test('Actual retry predicate retries JSON-RPC access denial three times', () => {
     const e = Object.assign(new Error('thread access denied'), {code: -32003});
     assert.deepEqual([0,1,2,3].map(n => retry(n,e)), [true,true,true,false]);
   });
   test('Actual retry predicate stops for numeric HTTP 403', () => {
     assert.equal(retry(0, {status: 403}), false);
   });
   test('Actual periodic-refetch condition accepts unclassified denial again at attempt zero', () => {
     assert.equal(retry(0, {code: -32003, message: 'thread access denied'}), true);
   });
   const begin = main.indexOf('var tR=8,nR=5');
   const classOffset = main.indexOf('vR=class', begin) + 3;
   let n = acorn.parseExpressionAt(main, classOffset, {ecmaVersion: 'latest'});
   if (n.type === 'SequenceExpression') n = n.expressions[0];
   const schedulerContext = {
     setTimeout: () => ({fakeTimer: true}), clearTimeout: () => {}, Date,
   };
   schedulerContext.yR = vm.runInNewContext('(' + func(main, 'yR') + ')', schedulerContext);
   schedulerContext.bR = vm.runInNewContext('(' + func(main, 'bR') + ')', schedulerContext);
   const context = vm.createContext(schedulerContext);
   vm.runInContext(main.slice(begin, n.end) + '; globalThis.Scheduler = vR;', context);
   // The actual functions must close over the same context as the shipped constants.
   vm.runInContext(func(main, 'yR') + ';' + func(main, 'bR') + ';', context);
   const Scheduler = context.Scheduler;
   function harness() {
     const s = new Scheduler(); const dispatched = [], rejected = [];
     let serial = 0;
     return {s, dispatched, rejected, add(method, priority, source) {
       const id = String(++serial);
       s.enqueue({requestId:id, method, params:{}, scheduling:{priority, source},
         dispatch: () => dispatched.push({id, method, priority}),
         reject: reason => rejected.push({id, method, priority, ...reason})});
       return id;
     }};
   }
   test('Actual scheduler permits only three plugin metadata requests in flight', () => {
     const h = harness(); for (let n=0;n<10;n++) h.add('plugin/installed','background','plugins');
     assert.equal(h.dispatched.length,3); assert.equal(h.s.getQueueState().queuedBackgroundRequestCount,7);
   });
   test('plugin/installed is not coalesced; 132 identical requests fill 128 background slots and reject one', () => {
     const h = harness(); for (let n=0;n<132;n++) h.add('plugin/installed','background','plugins');
     assert.equal(h.dispatched.length,3); assert.equal(h.s.getQueueState().queuedBackgroundRequestCount,128);
     assert.equal(h.rejected.length,1); assert.equal(h.rejected[0].reason,'queue-full');
   });
   test('A background-saturated queue still dispatches critical requests into reserved capacity', () => {
     const h = harness(); for (let n=0;n<132;n++) h.add('plugin/installed','background','plugins');
     h.add('account/read','critical','account');
     assert.equal(h.dispatched.at(-1).method,'account/read'); assert.equal(h.s.getQueueState().inFlightRequestCount,4);
   });
   test('Critical fan-out fills eight in-flight slots plus sixteen queued critical slots before rejection', () => {
     const h = harness(); for (let n=0;n<3;n++) h.add('plugin/installed','background','plugins');
     for (let n=0;n<22;n++) h.add(n%2?'getAuthStatus':'account/read','critical','account');
     assert.equal(h.s.getQueueState().inFlightRequestCount,8);
     assert.equal(h.s.getQueueState().queuedCriticalRequestCount,16);
     assert.equal(h.rejected.length,1);
   });
   async function hydrationProbes() {
     const globals = {Y: x => x, Error};
     const c = vm.createContext(globals);
     vm.runInContext('var LNt=' + expression(shared, shared.indexOf('LNt=')+4) + ';' +
       func(shared, 'INt') + ';' + func(shared, 'FNt') + ';' + func(shared, 'OP') + ';', c);
     const begin = shared.indexOf('readThread(e,{includeTurns:t=!1');
     const end = shared.indexOf('async hydrateCollabThreads', begin);
     const method = vm.runInContext('({' + shared.slice(begin,end) + '}).readThread', c);
     async function requests(error, access) {
       let calls = 0;
       const store = {conversations:new Map(), threadReadStates:new Map(),
         backgroundThreadLookups:{delete(){}},
         params:{hostId:'durable',requestClient:{sendRequest:async()=>{calls++;throw error;}}}};
       for(let n=0;n<5;n++) {
         try {await method.call(store,'synthetic-thread',{requestOptions:{priority:'background',
           source:'thread_hydration', ...(access?{threadAccess:access}:{})}});} catch {}
       }
       return calls;
     }
     const denied = Object.assign(new Error('thread access denied'),{code:-32003});
     const missing = Object.assign(new Error('thread not found: synthetic-thread'),{code:-32600});
     assert.equal(await requests(denied),5);
     results.push({name:'Actual durable read method repeats denied requests on every cycle',pass:true});
     assert.equal(await requests(missing),1);
     results.push({name:'Actual durable read method caches normalized thread-not-found as a terminal control',pass:true});
     assert.equal(await requests(denied,'shared'),5);
     results.push({name:'Explicit shared background access still repeats denied requests',pass:true});
     fs.writeFileSync(path.join(dir, 'background-probe-results.json'), JSON.stringify(results,null,2)+'\n');
     console.log(JSON.stringify(results,null,2));
   }
   hydrationProbes().catch(e=>{console.error(e);process.exitCode=1;});
   
   '''
   #!/usr/bin/env python3
   import json,struct,subprocess,tempfile,sys
   from pathlib import Path
   app=Path(sys.argv[1] if len(sys.argv)>1 else '/Applications/ChatGPT.app')
   assets={'source/shared.js':'webview/assets/app-shared-8340f229c2b3.js','source/main-CwxgTOle.js':'.vite/build/main-CwxgTOle.js','source/main.js':'.vite/build/src-DCNiwad_.js'}
   with tempfile.TemporaryDirectory(prefix='codex-offline-probe-') as tmp:
    root=Path(tmp);(root/'source').mkdir()
    with (app/'Contents/Resources/app.asar').open('rb') as f:
     header=struct.unpack('<4I',f.read(16));tree=json.loads(f.read(header[3]));base=8+header[1]
     for target,name in assets.items():
      node=tree
      for part in name.split('/'):node=node['files'][part]
      f.seek(base+int(node['offset']));(root/target).write_bytes(f.read(node['size']))
    (root/'probe.cjs').write_text(HARNESS)
    subprocess.run(['node','--expose-internals',str(root/'probe.cjs')],check=True)

   The abandonment probe can be reproduced by extracting abandonDestination(e,t){...} through immediately before complete(e){ from the main asset, evaluating it as an object method, and calling with a synthetic window destination, startedAtMs:1, empty coalesced map, retainUnknownOutcome:()=>false, and requestScheduler.dropUnsentRequest:()=>false. Assert pending remains for the three ordinary reads and is removed for the two controls. This isolates the shipped abandonment branch; it is not a simulated backend hang.
4. sivdead commented on Oct 17, 2026

   

   sivdead

   on Oct 17, 2026

   More actions

   Additional macOS reproduction on a newer build: Send becomes unresponsive while the desktop app-server scheduler remains at 7–8 in-flight requests. A manual full app restart restores responsiveness.

   Environment

   • Desktop app: 26.1013.40934, build 20770 (incident launch logs also report build 20770).
   • Bundled backend: codex-cli 0.163.0-alpha.10.2.
   • Separately installed CLI: codex-cli 0.163.0.
   • macOS 27.0.1, build 26A434, Apple Silicon arm64.
   • The desktop updater reported up_to_date when checked after this incident.
   • Incident date: 2026-10-17. All timestamps below are UTC.

   Observed behavior and timeline

   1. The desktop app continued running; clicking Send in an existing long conversation appeared to do nothing. The draft remained in the composer. We have not established whether that particular click generated a transport request.
   2. The same desktop process logged malformed stdout JSONL messages starting at 03:39:09.737Z. The retained launch log contains 53 such parse errors. This is temporal correlation, not proof that parsing errors caused the scheduler stall.
   3. At 04:11:00.928Z, background thread/list requests were already expiring with seven requests in flight. Queue-expiry reports continued for over 90 minutes.
   4. At 05:35–05:36Z, history requests for the affected conversation expired after 60 seconds while 18 interactive requests waited. By 05:36Z, eight requests were in flight. Model, permission, hooks and config requests also timed out or expired during the stall.
   5. At 05:45:49.001Z, the queue still reported eight in-flight requests and 18 waiting interactive requests.
   6. At 05:49:42Z, the user explicitly quit the app, then relaunched it around 05:50Z. That shutdown is a manual recovery action, not evidence of an automatic crash/reload. After relaunch, history/item requests for the same conversation returned in milliseconds.

   Selected redacted log excerpts
   The following preserve the diagnostic fields. Conversation/request IDs, local paths, process IDs, window IDs, business domains and unrelated fields are omitted. No raw conversation content is included.

   2026-10-17T03:39:09.737Z error [StdioConnection]
     Failed to parse app-server stdout JSONL message
     errorMessage="Expected ':' after property name in JSON at position 65538 (line 1 column 65539)"
     errorName=SyntaxError
   
   2026-10-17T04:11:00.928Z warning [AppServerConnection]
     app_server_client_request_queue_rejected
     method=thread/list source=thread_list priority=background
     inFlightRequestCount=7 queuedBackgroundRequestCount=1
     queuedCriticalRequestCount=0 queuedInteractiveRequestCount=1
     queuedPriorityRequestCount=1 queueWaitMs=30001 reason=queue-expired
   
   2026-10-17T05:35:20.950Z warning [AppServerConnection]
     app_server_client_request_queue_rejected
     method=thread/turns/list source=tail_history priority=background
     inFlightRequestCount=7 queuedBackgroundRequestCount=3
     queuedCriticalRequestCount=0 queuedInteractiveRequestCount=18
     queuedPriorityRequestCount=3 queueWaitMs=59999 reason=queue-expired
   
   2026-10-17T05:35:20.951Z error [electron-message-handler]
     Request failed method=thread/turns/list durationMs=60000
     error={"code":-32001,"message":"App server request expired while queued"}
   
   2026-10-17T05:36:21.955Z warning [AppServerConnection]
     app_server_client_request_queue_rejected
     method=thread/turns/list source=tail_history priority=background
     inFlightRequestCount=8 queuedBackgroundRequestCount=1
     queuedCriticalRequestCount=1 queuedInteractiveRequestCount=18
     queuedPriorityRequestCount=1 queueWaitMs=60001 reason=queue-expired
   
   2026-10-17T05:37:38.747Z error [StdioConnection]
     Failed to parse app-server stdout JSONL message
     errorMessage="Expected ':' after property name in JSON at position 65526 (line 1 column 65527)"
     errorName=SyntaxError
   
   2026-10-17T05:45:49.001Z warning [AppServerConnection]
     app_server_client_request_queue_rejected
     method=configRequirements/read source=config priority=background
     inFlightRequestCount=8 queuedBackgroundRequestCount=1
     queuedCriticalRequestCount=1 queuedInteractiveRequestCount=18
     queuedPriorityRequestCount=1 queueWaitMs=30001 reason=queue-expired
   

   Read-only comparison after recovery
   We launched separate temporary app-server processes for the bundled backend and the separately installed CLI. For each, we initialized the protocol and sent three concurrent thread/turns/list requests for the same persisted conversation with limit=1 and itemsView=full. Each response was 368,183 bytes and valid JSON. All three completed in approximately 0.16 seconds total with the bundled backend and 0.23 seconds with the separate CLI. The temporary processes were then stopped. No turn/start, conversation deletion or history modification was performed.

   This does not reproduce or disprove the intermittent failure, nor prove that changing backend versions fixes it. It does show that the persisted history was readable through an independent protocol connection after recovery.

   Potential recovery gap to investigate
   Read-only inspection of this installed desktop build suggests that markSentToTransport clears the scheduler entry's expiration timer. complete removes an entry after a response, while expireRequest does not expire an already-sent entry. The stdout JSONL parse-error handler logs the failure but does not itself identify/complete the affected RPC.

   Please verify whether a lost/unparseable response can therefore leave a dispatched entry occupying a slot indefinitely, and whether caller-side timeouts release or cancel that same scheduler entry. We could not correlate an individual malformed message with a specific permanently occupied request, so this remains a hypothesis rather than a proven root cause.

   Requested behavior

   • Bound the lifetime of dispatched RPCs and define cancellation/slot release semantics without blindly retrying operations that may have side effects.
   • Recover an unhealthy transport/backend while preserving conversations and clearly reporting interrupted work.
   • Keep essential Send/model/config controls from indefinitely waiting behind stale requests.
   • Log the method, age and cancellation state of occupied slots (without user content) so this failure can be diagnosed directly.

   Related signature: #38385 describes a similar scheduler recovery gap on Windows. This comment adds macOS build 20770 evidence; a shared root cause is not yet established. No deterministic clean-profile reproducer is available.

   Submitted by Codex at the user's explicit request. Only selected sanitized technical evidence is included; no full logs, session IDs, screenshots, credentials or conversation contents are uploaded.