Skip to content

macOS Computer Use: fresh Safari state succeeds, then coordinate click fails with -10005 noWindowsAvailable after app restart #36459

Open

Description

@rootsid-GovernSafe

Summary

On macOS, Computer Use can successfully return a fresh full screenshot and accessibility tree for the visible Safari window, and accessibility-index plus keyboard actions work, but a coordinate click derived from that same screenshot immediately fails with:

SkyComputerUseError: Computer Use server error -10005: noWindowsAvailable

The first occurrence was reproduced immediately after quitting and reopening the ChatGPT desktop app. A fresh reproduction about 30 hours later produced the same split-path result. The failure is not tied to a hidden, narrow, off-screen, or multi-display Safari window.

Environment

  • ChatGPT desktop app: 26.727.40816 (bundle build 6067), bundle ID com.openai.codex
  • Computer Use JS client: @oai/sky 0.6.2
  • Signed helper: com.openai.sky.CUAService 26.727.1000550 (build 1000550), Developer ID team 2DC432GLL2; strict signature validation passed
  • macOS: 26.5.1 (25F80), Apple Silicon arm64
  • Active display topology: one built-in display, 1512 x 945 @ 120 Hz, main/online, CoreGraphics bounds (0,0) 1512x945
  • Safari test window: one normal large window; screenshot 1386x768; no full-screen, Split View, tiling, negative position, or geometry recovery was required
  • Helper count: 1 active SkyComputerUseService
  • Main ChatGPT app-server direct node_repl children not belonging to this reproduction: 7 stale candidates, aged approximately 35 minutes to 28 hours at capture; none were killed
  • System-wide node_repl: 9; stdio app-server: 2; SkyComputerUseClient mcp: 0
  • Zombie processes: 0 system-wide, 0 under the ChatGPT GUI parent, 0 under its main app-server

Exact reproduction

  1. Keep an existing normal Safari window open. Inventory all existing tabs; do not close or overwrite them.

  2. Request a fresh full state:

    await sky.get_app_state({ app: "com.apple.Safari", disableDiff: true })
  3. Confirm the response contains both a screenshot and accessibility tree for the same visible Safari window.

  4. Choose a benign blank point inside the page viewport, away from browser chrome and site controls. In this run the first verified point was (865, 570).

  5. Execute:

    await sky.click({ app: "com.apple.Safari", x: 865, y: 570 })
  6. Observe the immediate exception:

    SkyComputerUseError: Computer Use server error -10005: noWindowsAvailable
    

The same error was reproduced at a second screenshot-verified page-content point (975, 250).

Recovery matrix

All six progressive recovery methods were completed without bypassing authentication, changing permissions, restarting Safari/WindowServer, or changing external state:

  1. Waited, then fetched a fresh full Safari state: screenshot and accessibility tree succeeded; coordinate click failed -10005.
  2. Pressed Escape to dismiss any overlay, then fetched a fresh full state: both succeeded; coordinate click failed -10005.
  3. Captured and visually inspected a new screenshot from the large on-screen window, then used a new page-content coordinate: failed -10005.
  4. Restored keyboard focus with Tab, then fetched a fresh full state: both succeeded; coordinate click failed -10005.
  5. Opened the exact URL in one fresh task-owned tab, verified the same pre-existing signed-in account and loaded state, dismissed the account overlay, and fetched a fresh full state: coordinate click failed -10005.
  6. Captured one final screenshot from the recovered task tab and made a final screenshot-guided page-content click: failed -10005.

The task-owned tab was closed afterward. Focus returned to the pre-existing Analytics tab, the window remained large, and the original two tabs remained open. No composer opened, no text was typed into a site, and no public action occurred.

Expected behavior

A coordinate action should resolve and target the same normal Safari window that the immediately preceding full-state request successfully captured. A healthy screenshot/accessibility request should not be followed by noWindowsAvailable for the same app/window.

Bounded diagnostic evidence

Client-side exception prefix:

SkyComputerUseError: Computer Use server error -10005: noWindowsAvailable
  at @oai/sky/.../targets/mac/native-pipe.js

A bounded unified-log query covering 2026-08-01 19:34:30–19:42:30 AEST for SkyComputerUseService, ChatGPT, ComputerUseIPC, noWindowsAvailable, timeoutReached, AccessibilitySupport, and ScreenCaptureKit returned no matching records. This is recorded as a zero-result query rather than expanding the collection window or attaching unrelated logs.

Official update observed locally

Sparkle has staged a correctly signed ChatGPT app update 26.727.51351 (bundle build 6119) while 26.727.40816 (build 6067) remains installed. The staged app still contains @oai/sky 0.6.2 and helper 26.727.1000550, identical to the installed Computer Use components. The update was not installed, copied, launched, or modified; its fix status is unknown.

Related reports

This report may share lifecycle or target-correlation code with those issues, but adds the deterministic state-success/action-failure split and the exact -10005 noWindowsAvailable result.

Activity

  1. github-actions commented on Aug 1, 2026

    @github-actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  2. rootsid-GovernSafe commented on Aug 11, 2026

    @rootsid-GovernSafe
    Author

    Update from a newer bundled build, reproduced 2026-08-11:

    • ChatGPT host 26.803.41515
    • Computer Use plugin 1.0.1000633
    • bundled @oai/sky 0.6.2
    • On a large normal Safari window, a fresh full screenshot and accessibility state succeeded.
    • Coordinate page scroll and screenshot-guided clicks still failed before Safari received a pointer event with SkyComputerUseError: Computer Use server error -10005: noWindowsAvailable.
    • Element-index activation changed the visible Repost control from collapsed to expanded, but no share sheet painted and accessibility exposed neither Repost with your thoughts nor Repost now.
    • Six distinct recoveries were exhausted: wait plus fresh state; dismiss overlays; fresh screenshot and content coordinate; keyboard focus and navigation; exact URL in a fresh task tab; final screenshot-guided attempt.
    • Task-created browser state was closed. No composer, copy entry, share selection, or public action occurred.

    This shows plugin 1.0.1000633 also does not resolve the state-success/action-failure split reported here. Can the Computer Use maintainer confirm whether a build newer than 1.0.1000633 contains the Safari window-to-pointer targeting fix, or provide the installable build/version that should be retested?

  3. niyoufei commented on Sep 3, 2026

    @niyoufei

    Additional independently observed reproduction on build 26.831 (2026-09-03)

    Adding a newer-build reproduction with the same error and read/coordinate-action split, observed in Figma and System Settings. This may be related to this issue and the cross-app report #38508; a common internal root cause has not been established.

    Observed failure

    Public native CUA coordinate clicks return:

    Error: Computer Use server error -10005: noWindowsAvailable
    

    Native AX and screenshots remain readable. Semantic AX clicks successfully navigate System Settings and open a Figma top-level Help menu. A subsequent read confirmed that menu was closed without invoking any menu command.

    Environment

    • macOS 26.5.1, build 25F80, Apple silicon.
    • ChatGPT desktop 26.831.21537, build 7579.
    • Codex Computer Use native helper bundle build 1000926.
    • Native helper and bundled client passed strict code-signature verification.
    • System Settings showed Codex Computer Use enabled under both Accessibility and Screen & System Audio Recording. No permission settings were changed.

    Reproduction evidence

    Targets were identified from public AX/screenshot observations. Screenshot dimensions were independently verified during the investigation from JPEG SOF markers in the bytes returned by public screenshot methods; not every earlier attempt included an advance raw-dimension check. The returned images were JPEG, despite a PNG-oriented wrapper/example.

    The following table summarizes distinct target cases, not the complete chronological attempt log. Some targets were tested again after a specific recovery intervention, as described below.

    Target Screenshot dimensions Safe coordinate Result
    Figma, empty title-bar region 1228 × 768 (900, 16) -10005 / noWindowsAvailable
    System Settings, empty title-bar region 723 × 625 (600, 26) Same error
    Figma, already-selected page row in content area 1228 × 768 (130, 163) Same error

    Coordinates are specific to these observed windows, not reusable instructions. No document-editing or permission-toggle controls were clicked. This is not an NSOpenPanel reproduction: no file picker was opened in this investigation.

    Bounded recovery attempts

    1. Fresh app binding by Figma bundle identifier: AX/screenshot succeed, coordinate failure persists.
    2. Figma Window menu → existing document entry (makeKeyAndOrderFront): semantic action succeeds; coordinate failure persists.
    3. One graceful termination of the identity-checked native helper, followed by a normal public CUA request: a new helper starts and public reads reconnect successfully. Figma and the ChatGPT main application were not restarted. Coordinate failure persists.
    4. One public REPL reset after the helper replacement: native APIs and reads are available in the new client session; content-area coordinate failure persists.

    Latest exact reproduction: 2026-09-03 00:22:39.178–00:22:39.205 UTC. A single content-area click in the fresh helper/client combination returned the error. The subsequent AX/screenshot showed the original document and page, without a modal or visible change.

    Static boundary checks

    The bundled JavaScript maps click([x,y]) to numeric x/y, preserves the policy-resolved app target, and forwards a coordinate action. No JavaScript-level field loss or resampling defect was identified. The bundled type comment defines x/y as app-window screenshot coordinates. No native window ID or display-selection option is exposed by the public App binding inspected here.

    These checks do not establish the native implementation's exact failing branch. The failure should not be diagnosed as missing macOS permission merely from its name.

    Requested investigation

    Please inspect native coordinate-action window discovery/matching versus semantic AX targeting. Coordinate clicks failed in Figma and System Settings. After rebuilding the native helper and resetting the public REPL client, a Figma content-area coordinate click still returned the same error. Please advise a supported component fix or a build known to address this failure, with an acceptance check using a real coordinate click and observable UI response.

    Privacy and scope

    This report omits user names, local workspace paths, document/file identifiers, thread IDs, screenshots, business data, pairing codes, tokens, and credentials. It contains no full session transcript. No plugin was imported or run, no companion was started, and no business task/model operation occurred during these tests.

  4. ozzyozbourne commented on Sep 4, 2026

    @ozzyozbourne

    Additional reproduction observed September 3–4, 2026, using macOS Calculator as a minimal target.

    Read succeeds, coordinate click fails

    1. Select Calculator through cua.getApp("Calculator").
    2. Accessibility state and a window screenshot succeed.
    3. Use the current accessibility element numbers to activate Clear → 2 → + → 3 → Equals. Both accessibility text and the screenshot confirm 5.
    4. Request a coordinate click inside the visible AC button using app.click([x, y]).
    5. The tool returns Computer Use server error -10005: noWindowsAvailable.

    In the observed 230 × 408 screenshot, the AC center was approximately (88, 157). Nearby points inside the same button also failed. These coordinates are specific to that screenshot, not reusable targets.

    Desktop and terminal CLI

    The same Calculator read/accessibility-success versus coordinate-failure split was reproduced from the desktop task and from a standalone codex exec --yolo --ephemeral task. The terminal CLI reported 0.153.1 and the desktop bundled executable reported 0.153.0 when checked during the investigation. The CLI retry could access Calculator and complete 2 + 3 through accessibility, but coordinate activation still failed. This does not establish which component contains the defect.

    Window restoration observation

    With Chrome minimized, selecting the app returned noWindowsAvailable, and requesting its screenshot through the prior binding returned timeoutReached. The human observer saw Chrome restore to its previous window size and take focus during those requests, although no screenshot was returned. Thus an error did not imply that no UI side effect occurred.

    Fresh bindings and re-enabling the Computer Use MCP server did not resolve the observed coordinate failure. Separately, a local Flutter app has intermittent accessibility-action failures; we are not treating those as proven to share this root cause.

    Could maintainers provide a supported recovery procedure or a build that fixes coordinate targeting? A useful acceptance check is an observable Calculator coordinate click, not merely successful screenshots or accessibility reads.

    No private paths, screenshots, credentials, or raw session logs are included.

  5. TheSeamau5 commented on Sep 6, 2026

    @TheSeamau5

    running into this same issue
    Computer Use server error -10005: noWindowsAvailable

  6. rayshineeeee commented on Sep 6, 2026

    @rayshineeeee

    same issue here

  7. perpetua-engineering commented on Sep 30, 2026

    @perpetua-engineering

    Additional reproduction on September 30, 2026, in macOS System Settings, with the same screenshot/AX-success versus coordinate-input-failure split.

    Environment

    • ChatGPT/Codex desktop app: 26.928.21956, bundle build 12404, bundle ID com.openai.codex (read from the installed app after the operator restarted it).
    • macOS 27.0, build 26A428, Apple Silicon (arm64).
    • Public cua_repl API, targeting com.apple.systempreferences.

    Minimal reproduction

    1. Open System Settings with a normal window. In our test it was on Software Update.
    2. Bind the app:
      let app = await cua.getApp('com.apple.systempreferences');
      The accessibility tree is returned successfully.
    3. Capture a fresh screenshot:
      await app.getScreenshot();
      The expected System Settings window is captured successfully.
    4. Identify the Appearance sidebar row in that screenshot and call await app.click([x, y]) at its center. Our post-restart screenshot was 1480 × 1818 and the observed point was [174, 974]; these coordinates are specific to that screenshot and must be recomputed for other window layouts.
    5. The coordinate action immediately returns:
      Computer Use server error -10005: noWindowsAvailable
      
      The post-restart tool call reported 0.0461 seconds.
    6. Fetch a fresh full accessibility tree and click the Appearance row by its current element index. This succeeds: the selected sidebar row and window title change to Appearance.
    7. Fetch the resulting state and use the current accessibility index to return to Software Update. This also succeeds.

    No preference was changed, no update was initiated, and the original Settings page was restored.

    Recovery results

    The coordinate failure persisted after each of:

    • Resetting the public Computer Use REPL session and obtaining a fresh app binding/screenshot.
    • Invoking the window's exposed Raise secondary action and taking a fresh observation.
    • The operator restarting the desktop app, followed by a fresh binding and screenshot in the new session.

    Accessibility-index navigation continued working after the failed coordinate clicks. This minimal reproduction required no Simulator or application-under-test interaction. It does not establish the native root cause or whether every application is affected.

    Could maintainers investigate coordinate-action window resolution versus accessibility targeting, and identify a supported recovery or fixed build? An acceptance check should include a screenshot-guided coordinate click with a verified visible transition, not only successful capture/AX reads.

    No screenshots, account identifiers, private project details, local paths, or raw session logs are attached.

  8. SandroMaglione commented on Oct 3, 2026

    @SandroMaglione

    Additional cross-app reproduction on October 3, 2026

    I am still affected by Computer Use server error -10005: noWindowsAvailable. It prevents practical Computer Use workflows in Blender and drawing apps. The failure has been present for at least several weeks. Local test records from September 5 also document coordinate failures in CLIP STUDIO PAINT and TextEdit.

    This report adds a fresh Calculator comparison, TextEdit reproduction, and a Blender activation/fullscreen recovery matrix. It may also relate to #38508.

    Environment verified on October 3

    Component Value
    macOS 26.3, build 25D125
    Platform Darwin 25.3.0 arm64 arm, Apple M1
    Installed ChatGPT/Codex desktop bundle 26.928.20755, build 12246, bundle ID com.openai.codex
    Current test host T3 Code Alpha 0.0.45 through the Codex harness
    Standalone Codex CLI 0.160.0
    CLI bundled with ChatGPT 0.159.0
    Bundled Computer Use JS client @oai/sky 0.7.5
    Installed Computer Use helper 26.924.1001281, build 1001281, bundle ID com.openai.sky.CUAService
    Blender 4.5.14 LTS
    Display One built-in display, 1440 × 900 logical at 60 Hz, 2880 × 1800 pixels; mirroring off

    The current reproduction used the public cua_repl API. The ChatGPT GUI was not running during these fresh tests; its bundled Computer Use runtime was in use. Please keep this distinction when triaging the host.

    Earlier in this same investigation, System Settings visibly showed Codex Computer Use enabled in both Accessibility and Screen Recording. No permission settings were changed. Strict deep code-signature verification of the installed helper passed again today. Its executable matches the helper bundled with ChatGPT, with SHA-256:

    d4b1775138342c0df8e9451df3c23dda3fdef44c4ade50a2de382c188cf61b54
    

    Fresh minimal reproduction in Calculator

    1. Select Calculator using await cua.getApp("com.apple.calculator").
    2. Read its accessibility tree and screenshot. Both succeed, and the display is 0.
    3. Click the center of the visible 2 button using screenshot coordinates.
    4. The coordinate action fails immediately.
    5. Click that same button by its current accessibility element index. This succeeds. The AX value and screenshot both show 2.
    6. Clear via the current AX element. The display returns to 0.

    In this specific 460 × 816 screenshot, the button center was [176, 639] and the returned AX index for Two was 18. Both must be derived again for other sessions.

    let calculator = await cua.getApp("com.apple.calculator");
    await calculator.getAXStateAndScreenshot();
    await calculator.click([176, 639]);
    // Computer Use server error -10005: noWindowsAvailable
    
    // After checking the current AX tree:
    await calculator.click(18);
    await calculator.getAXStateAndScreenshot();
    // AX text and screenshot both show 2

    Observed timestamps in UTC:

    • Coordinate click failure: 2026-10-03T10:03:54.668Z, tool duration about 0.024 seconds.
    • AX click success: 2026-10-03T10:03:59.470Z.
    • Coordinate failure repeated after a fresh screenshot: 2026-10-03T10:05:18.181Z.

    Cross-app evidence

    Target Observed result
    Blender, normal window Screenshots/AX work; coordinate click and scroll fail in the broken state.
    Blender after a human title-bar click in a normal window Coordinate workspace clicks, viewport zoom scrolling, and editor-divider dragging all worked with visible changes. Divider width and zoom were restored.
    Blender while I subsequently used other apps Coordinate click and scroll failed again. A fresh app binding did not repair the next click. This does not isolate the exact focus/Space transition that invalidates control.
    Blender in macOS fullscreen after another human click Fresh 2880 × 1800 screenshot succeeded; Layout-tab click at [565, 32] and viewport scroll both failed with the same error.
    Calculator Fresh screenshot/AX succeed; coordinate Two click fails; AX activation of Two succeeds visibly.
    TextEdit, fresh October 3 test AX/screenshot of its Open dialog succeed; coordinate Cancel click at [1425, 835] in a 1760 × 900 screenshot fails at 2026-10-03T10:03:32.808Z. This fresh reproduction used an NSOpenPanel, not the text canvas.
    CLIP STUDIO PAINT and TextEdit, September 5 local test records Clip Studio screenshots, AX/menu and keyboard actions worked; coordinate click/drag failed. TextEdit AX click and typing worked; coordinate click failed. These are historical records, not fresh Clip Studio tests today.

    The Mac was unlocked. A one-time manual activation did not establish reliable continued background control. The temporary normal-window recovery resembles the observation in this comment, but fullscreen plus another manual click did not recover control here.

    Other Blender recovery attempts in this investigation failed: fresh binding, keyboard activation, AX Raise, the native Window menu document entry, native Zoom, REPL reset, and one graceful helper restart followed by reconnect. The later manual title-bar activation was the intervention after which visible coordinate actions succeeded.

    Additional debug evidence

    A fresh process inventory found one SkyComputerUseService, PID 87162, parent PID 1, two SkyComputerUseClient processes, and four node_repl processes. Multiple client processes are recorded as context, not a confirmed cause.

    A bounded unified-log query for the helper covering 19:03:25–19:04:10 JST returned no messages matching noWindowsAvailable, cgWindowNotFound, windowNotFound, or denied. ScreenCaptureKit window-enumeration messages were present in that interval. For example:

    19:03:49.947 SkyComputerUseService[87162] ScreenCaptureKit SCWindow:
    windowID=1509 windowTitle=<private> windowOwner=<private>
    isOnScreen=0 isActive=0
    frame=(270.000000,413.000000,230.000000,408.000000)
    
    19:04:00.697: same windowID, flags, and frame
    

    That frame is 230 × 408 logical points, the size of the captured Calculator image at 2× scale. The log hides the owner/title, so size alone does not establish that it is Calculator or prove a Space/filtering defect.

    Catching the repeated coordinate exception exposed only the public runtime wrapper frames. Temporary-directory prefixes have been removed:

    Error: Computer Use server error -10005: noWindowsAvailable
        at kernel.js:1595:16
        at Object.settle (worker-runtime.js:396:5)
        at kernel.js:1871:11
        at handleInputFrame (worker-runtime.js:418:7)
        at Socket.<anonymous> (worker-runtime.js:434:9)
    

    Requested investigation

    Please investigate native coordinate-action window discovery/matching and its handling of activation and macOS Spaces/fullscreen, compared with screenshot capture and AX activation. The precise native failing branch remains unknown.

    Could maintainers identify a fixed build or supported recovery procedure, and the specific diagnostic logging needed to resolve this? A useful acceptance test is an observable coordinate click, scroll, and drag, followed by continued control while the user works in another app. Successful screenshots alone do not verify recovery. The official Computer Use documentation describes scoped background work on macOS.

    No account credentials, private paths, artwork, screenshots, or full session logs are attached. Calculator was cleared after testing; no text document was opened or edited for the fresh TextEdit test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    appIssues related to the Codex desktop appbugSomething isn't workingcomputer-use

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions