Skip to content

app-server: resume ignores configuration overrides for unsubscribed settled systemError threads #54016

Open

Description

@zhuyankarl

Problem

An unsubscribed thread whose last turn failed can remain in systemError. thread/resume with changed configuration then returns the loaded thread while ignoring the requested overrides. This differs from the same operation on an unsubscribed idle thread and prevents clients that require verified configuration replacement from recovering independently of other active threads.

Reproduction

Reproduced with Codex 0.160.0 on Linux x86_64, using a temporary isolated home and a credential-free synthetic loopback Responses provider:

  1. Set provider request/stream retries to zero; leave the default thread unload delay.
  2. Start persistent threads A and B on the same app-server. Hold B's provider response open.
  3. Return HTTP 429 for A and await turn/completed with status: failed.
  4. Verify A is systemError; unsubscribe A and receive unsubscribed.
  5. Resume A with changed developer instructions or approval settings.

A remains loaded in systemError, and the changed settings are not applied. B remains active. Restarting the entire server is undesirable because it interrupts unrelated work.

Root-cause analysis

In codex-rs/app-server/src/request_processors/thread_processor.rs, the changed-configuration reload branch of resume_running_thread requires ThreadStatus::Idle in addition to no subscribers, direct-input eligibility and a non-running agent. A settled failed thread satisfies the other conditions but has ThreadStatus::SystemError.

Potential fix: allow Idle | SystemError in this branch, retaining all existing ownership/running-state guards and requiring confirmed shutdown before removing the loaded thread. A useful regression holds B active, verifies changed settings and preserved history for unsubscribed A, and verifies that a still-subscribed A keeps its existing settings.

Supported workaround and validation limits

The same isolated binary probe with thread_unload_delay_secs=0 produced systemError -> notLoaded -> idle for A while B stayed active. Clients must await the actual notLoaded notification; the unsubscribe response alone is not proof of teardown.

No modified native binary has been deployed. A proposed source change and real-protocol regression were reviewed locally, but Rust compilation/tests were not run because Cargo/rustfmt are unavailable in the reporting environment. This report follows the repository policy requesting issues instead of external PRs.

Related downstream context: openclaw/openclaw#148837 and openclaw/openclaw#149000.

All messages, responses, and state used for reproduction are synthetic. No production logs, credentials, account identifiers, conversation history, or host configuration are included.

Activity

  1. added
    bugSomething isn't working
    app-serverIssues involving app server protocol or interfaces
    on Oct 14, 2026
  2. rksharma-owg commented on Oct 15, 2026

    @rksharma-owg

    I independently reproduced this on current main at ac9db7d, using an owned stdio app-server, a temporary home, and a synthetic loopback Responses provider. No account or production model requests were involved.

    A reached systemError after HTTP 429. After thread/unsubscribe, resuming A with approvalPolicy: on-request returned never, while B remained active. The same probe with the settled-error reload guard adjusted returned on-request and left B active. A separate Rust integration regression also checks that a still-subscribed A retains its settings and that failed-turn history survives the reload.

    The local checks use Rust 1.98.0, not the repository's pinned 1.95.0; this is not full CI validation. I have not opened a PR, in line with the contribution policy.

    Would the team prefer settled systemError threads to use the existing idle-cache reload path, retaining the subscriber, running-state, ownership and confirmed-shutdown guards, or should callers continue waiting for an explicit notLoaded receipt? I can provide the focused reproduction evidence here in the format maintainers prefer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    app-serverIssues involving app server protocol or interfacesbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions