Skip to content

Windows sandbox cannot recover when deny_read_acl_state.json becomes 22 NUL bytes after a system crash #34841

Open

Description

@sajidfarooq

Product and environment

  • Codex desktop app on native Windows
  • Observed app package: OpenAI.Codex_26.715.8383.0_x64
  • Windows 11 25H2, build 26200.8875
  • Sandbox mode: ordinary native-Windows workspace-write execution
  • State path: %USERPROFILE%\.codex\.sandbox\deny_read_acl_state.json

Problem

After a Windows kernel crash, the Codex Windows sandbox can become persistently unusable because deny_read_acl_state.json contains exactly 22 zero bytes rather than JSON. Every ordinary sandbox setup tries to deserialize or reconcile this state and fails before the requested command process starts. Execution outside the sandbox still works because it bypasses this setup path.

This has now occurred three times on the same machine. Renaming the malformed file allows the next ordinary sandbox setup to regenerate valid state immediately, confirming that the requested command and repository are not the cause.

The upstream machine crashes are a separate Microsoft Wof.sys issue. This Codex report is about durability and recovery: an external crash can leave a reconstructible state artifact malformed, and Codex then turns that artifact into a persistent outage for every future sandboxed command.

Observed error and file contents

Ordinary commands fail before PowerShell or the requested process starts:

windows sandbox: helper_unknown_error: apply deny-read ACLs

The malformed file is:

Length: 22 bytes
Hex:    00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00

The intended empty serialized state is also 22 bytes:

{
  "principals": {}
}

Reproduction using captured state

  1. Place a captured 22-NUL deny_read_acl_state.json at the state path.
  2. Run any ordinary sandboxed command from a valid workspace.
  3. Observe that sandbox setup fails before the requested process starts.
  4. Preserve and rename the malformed file.
  5. Run an ordinary sandboxed command again.
  6. Codex regenerates a valid 22-byte JSON file and the command starts successfully.

This recovery has been repeated after multiple crashes.

Evidence from three incidents

Incident State length State content Write-to-reboot interval Result after reboot
1 22 bytes all NUL ~20.6 seconds every ordinary sandbox setup failed
2 22 bytes all NUL ~17.3 seconds same
3 22 bytes all NUL 17.364 seconds same

The first two Windows minidumps independently identify the upstream crashes as bugcheck 0x139/A in Wof!WofPreDirectoryControlCallback+0x695, with the same Microsoft failure hash. The third Windows event is also 0x139/A; its dump has not yet been independently analyzed. Those kernel details establish the external-crash context, but they are not required to reproduce Codex's malformed-state failure.

Relevant implementation

The current Windows deny-read state implementation serializes with serde_json::to_vec_pretty and persists using a direct std::fs::write(path, bytes):

codex-rs/windows-sandbox-rs/src/deny_read_state.rs

The observed implementation does not provide:

  • same-directory temporary-file creation followed by atomic replacement;
  • explicit durable flushing before publication;
  • validation of the newly written file before replacing the prior state;
  • malformed-state quarantine and safe reconstruction;
  • an interprocess lock around state reconciliation.

Regardless of the upstream kernel fault, a malformed reconstructible state file should not permanently disable sandbox setup.

Expected behavior

  • A crash during or shortly after persistence should leave either the previous valid state or a complete new state.
  • If state is malformed, setup should quarantine it for diagnostics, reconstruct safe state, and continue or emit a targeted actionable diagnostic.
  • A corrupt reconstructible state artifact should not prevent every sandboxed command from launching indefinitely.

Requested fix

  1. Serialize and validate the complete next state in memory.
  2. Write it to a unique temporary file in the same directory.
  3. Flush the temporary file durably (sync_all or the appropriate Windows equivalent).
  4. Atomically replace the destination while preserving the previous valid file until publication.
  5. Coordinate writers with an interprocess mutex or equivalent lock.
  6. On deserialization failure, quarantine the malformed file and safely reconstruct state.
  7. Report a specific malformed-state diagnostic rather than the generic ACL-application error.

Privacy

This public report intentionally excludes usernames, repository names and paths, thread IDs, raw dumps, .codex databases, sandbox logs, credentials, and secrets. Sanitized debugger conclusions can be supplied publicly; raw evidence can be offered to OpenAI through a private channel if requested.

Activity

  1. added
    bugSomething isn't working
    appIssues related to the Codex desktop app
    windows-osIssues related to Codex on Windows systems
    sandboxIssues related to permissions or sandboxing
    on Jul 22, 2026
  2. github-actions commented on Jul 22, 2026

    @github-actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  3. sajidfarooq commented on Jul 23, 2026

    @sajidfarooq
    Author

    The primary addition this report provides is the “why it happens”. I have provided concrete evidence that it happens when windows crashes. I have also provided the fix. This should help prevent it in the future.
    I have separately provided Microsoft with the relevant report on the Windows crash.

  4. Spiralis commented on Jul 23, 2026

    @Spiralis

    Independent confirmation from another native Windows Codex Desktop machine.

    Environment

    • Windows: Microsoft Windows NT 10.0.26200.0
    • Current Codex app package: 26.715.10079.0
    • Codex app package recorded in the incident-period sandbox log: 26.715.3651.0
    • Sandbox: native Windows workspace-write/elevated backend

    Timeline and evidence

    All times below are local (Europe/Oslo) on 2026-07-18:

    • Around 01:40:13, the active Codex sandbox log output became NUL-filled while the helper was updating sandbox state.
    • deny_read_acl_state.json received a last-write timestamp of 01:40:15.
    • Windows then recorded an unexpected restart/Kernel-Power event and bugcheck around 01:40:45.
    • After reboot, every ordinary sandboxed process failed before the requested command started. The failure persisted across app restarts and app updates until 2026-07-23.

    The repeated error was:

    setup error: apply deny-read ACLs
    
    Caused by:
      0: parse deny-read ACL state <USERPROFILE>\.codex\.sandbox\deny_read_acl_state.json
      1: expected value at line 1 column 1
    

    The preserved malformed state has:

    Length:        22 bytes
    Non-zero bytes: 0
    SHA-256:       6A4875DDACEAA91FB3369F0F6D962F77442DAF1B1D97733457D12BCABDF79441
    

    It contains no paths, usernames, credentials, tokens, or other data—only 22 NUL bytes. The file can be supplied if useful, although the size, hash, and byte description fully characterize it.

    Recovery confirmation

    Moving only the malformed state file to a .bak path allowed the next ordinary sandbox setup to regenerate:

    {
      "principals": {}
    }

    setup_error.json disappeared, and ordinary sandboxed PowerShell, Git reads, rg, Node, and workspace writes immediately succeeded again. A disposable write probe was created and removed successfully.

    This independently confirms that:

    1. the failure is machine-global sandbox state, not a particular repository, drive, or worktree;
    2. a separate worktree cannot avoid it because setup fails before the requested workspace process starts;
    3. malformed-state quarantine/reconstruction is needed even if atomic replacement and writer locking prevent future corruption.

    I did not analyze the Windows minidump; the relevant Codex defect is the persistent outage caused by failure to recover reconstructible local state. This also corroborates the recovery behavior described in #34276.

  5. Spiralis commented on Jul 23, 2026

    @Spiralis

    Follow-up from the same Windows machine after recovering the corrupt ACL-state file: I hit a second, reproducible workspace-write failure that may help isolate the ACL refresh path.

    Inside a Codex-managed Git worktree declared as the writable root:

    1. An existing tracked file could be rewritten by a sandboxed formatter.
    2. A file newly created by Codex apply_patch in the same directory could be read but not rewritten by the same formatter:
    EPERM: operation not permitted, open '<worktree>\services\web-app\src\routes\index.tsx'
    
    1. icacls showed that the parent directory had an inheritable (OI)(CI)(M) workspace-grant ACE and the pre-existing file had the corresponding inherited (I)(M) ACE. The newly created/replaced file was missing only that workspace-grant ACE; its normal CodexSandboxUsers:(I)(M) and other inherited entries were present.
    2. A second tracked file in the same formatter invocation (one that existed before the session) formatted successfully.
    3. Re-running only the formatter outside the sandbox succeeded immediately.

    This looks consistent with apply_patch creating or atomically replacing a file with a DACL that does not retain the per-workspace grant marker, even though the parent ACE is inheritable. It also explains why using a separate worktree did not eliminate all sandbox failures: Git isolation was correct, but the newly created files themselves became unwritable to later sandboxed processes.

    I redacted machine-local account names and SIDs here; I can provide the full icacls before/after output if useful.

  6. ashjo42 commented on Aug 6, 2026

    @ashjo42

    Confirming the exact 22-NUL corruption and persistent recovery failure on a newer Codex Desktop build, with explicit user approval to share these sanitized diagnostics.

    Environment:

    • Windows 11 Pro, version 10.0.26200 (build 26200), x64
    • Codex Desktop MSIX: OpenAI.Codex_26.730.8199.0_x64
    • Custom CODEX_HOME on a local NTFS drive

    Current state:

    • .sandbox/deny_read_acl_state.json length: 22 bytes
    • All 22 bytes are 0x00
    • SHA-256: 6A4875DDACEAA91FB3369F0F6D962F77442DAF1B1D97733457D12BCABDF79441
    • Last write: 2026-08-04 14:15 local time
    • The failure persists across Codex Desktop restarts.

    The sandbox log repeatedly records:

    setup error: apply deny-read ACLs
    
    Caused by:
      0: parse deny-read ACL state <CODEX_HOME>\.sandbox\deny_read_acl_state.json
      1: expected value at line 1 column 1
    
    setup refresh: exited with status ExitStatus(1)
    

    On 2026-08-06 the same sequence recurred many times during command startup/refresh. The sandbox never recovers or quarantines/regenerates the reconstructible state file.

    User-visible impact in Desktop:

    • Ordinary sandboxed commands fail before the requested process starts.
    • The agent retries through escalation, producing a repeated approval-dialog loop.
    • Restarting the app does not recover it.
    • Unattended work becomes impossible.

    This machine also shows the permission-persistence symptom in #36497: after restart, active tasks are effectively workspace-write/on-request despite selecting Full access, which makes the corrupt-state retry path especially visible as repeated approvals.

    No credentials, transcripts, databases, usernames, thread IDs, or private project paths are attached.

  7. wunsir commented on Aug 29, 2026

    @wunsir

    Additional Windows reproduction on Codex Desktop 26.825.5331.0.
    After an unexpected Windows shutdown, %USERPROFILE%.codex.sandbox\deny_read_acl_state.json was exactly 22 bytes and every byte was 0x00.
    Every ordinary sandboxed command and apply_patch then failed with:
    setup error: apply deny-read ACLs Caused by: 0: parse deny-read ACL state 1: expected value at line 1 column 1
    Restarting Codex did not repair it. After preserving and renaming the malformed state file, then restarting Codex, the app regenerated a valid 22-byte JSON file. setup_error.json disappeared, ordinary sandbox commands succeeded, and an apply_patch write/read/delete check passed.
    The malformed file timestamp was within about one minute of the unexpected shutdown/reboot. The corrupted shared ACL state affected multiple tasks until the file was regenerated.
    Codex feedback ID: 01a04d88-e06d-7103-97eb-4bdec4a6f2cc

  8. jakobboss1010-source commented on Sep 1, 2026

    @jakobboss1010-source

    Refreshed current-main recovery candidate: jakobboss1010-source@6050e61 . The patch quarantines malformed deny_read_acl_state.json, reconstructs typed empty state, writes and durably flushes a same-directory temporary file, and atomically publishes it with ReplaceFileW or MoveFileExW while serializing the complete transaction with a Windows named mutex. Windows validation against upstream main 2b7c279: the captured 22-NUL regression, open-reader replacement, thread concurrency, and child-process concurrency tests all passed; scoped Clippy, Rust formatting, and diff checks passed. No live ACL fault injection was performed.

  9. landmatchthailand commented on Sep 6, 2026

    @landmatchthailand

    I am experiencing the same failure on Windows 10 Pro 22H2, OS Build 19045.6466, with desktop app version 26.901.51231.

    Error:
    helper_unknown_error: apply deny-read ACLs

    The sandbox log identifies:
    parse deny-read ACL state: deny_read_acl_state.json
    expected value at line 1 column 1

    Read-only verification confirmed:
    FILE_BYTES=22
    NUL_BYTES=22

    Restarting the app did not resolve the error. No state-file repair, ACL/config change, or sandbox bypass has been performed.

    Submitted feedback ID:
    no-active-thread-01a0785f-be4e-7953-bbec-28cc3a1a4015

    Could a maintainer confirm a supported recovery procedure that preserves the existing deny-read restrictions and sandbox boundaries?

  10. dfrer commented on Sep 9, 2026

    @dfrer

    Independent corroboration

    Independent corroboration from another affected native-Windows installation.

    Environment:

    • Windows 10 Home, build 19045, x64
    • Codex Desktop bundled CLI 0.153.4
    • Native Windows sandbox configured as elevated
    • State file: %USERPROFILE%\.codex\.sandbox\deny_read_acl_state.json
    • State length: 22 bytes
    • State contents: 22 NUL bytes
    • State SHA-256: 6a4875ddaceaa91fb3369f0f6d962f77442daf1b1d97733457d12bcabdf79441

    Observed failure:

    windows sandbox: helper_unknown_error: apply deny-read ACLs

    The configuration parses successfully. Two expected sandbox accounts remain present and enabled. Selected account identities and memberships, plus 19 accessible relevant ACL snapshots, remained stable during a read-only investigation. Two profile ACLs were inaccessible, so this is not a complete ACL certification. Three observed firewall-rule snapshots were stable; WFP was not verified.

    The malformed state was preserved without renaming or deletion. Source review indicates that possible setup paths may involve account credentials, protected account records, firewall/WFP state, inheritable ACL grants across fixed and dynamic roots, generated state files, and detached ACL work; these are possible source paths, not evidence that they all ran here. The reviewed path did not expose a reliable transaction-completion barrier.

    We could not establish a complete operational rollback boundary. In particular, provisioning can reset account passwords before replacing protected credential records; a login-retry branch may remove a generated account record before UAC; inheritable ACL changes may affect existing descendants; and the complete managed-object set is not documented sufficiently for safe local rollback. Malformed JSON alone is not proven to trigger the credential retry.

    For that reason, recovery was not attempted. Could OpenAI provide:

    1. The supported recovery/reset procedure for malformed deny_read_acl_state.json.
    2. The complete accounts, credential records, files, registry objects, firewall/WFP state, and ACL roots the recovery path may modify.
    3. The supported rollback procedure for partial or failed setup.
    4. A reliable completion signal for asynchronous ACL setup.
    5. The current status of a fix that quarantines malformed reconstructible state and regenerates it safely.

    A sanitized private diagnostic package can be supplied to OpenAI Support. It contains no credentials, raw configuration, private repositories, or personal project data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    appIssues related to the Codex desktop appbugSomething isn't workingsandboxIssues related to permissions or sandboxingwindows-osIssues related to Codex on Windows systems

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions