Skip to content

opencode-claimant: bound session that never received its prompt (task 6722) has no recovery path — stuck claimed forever #4758

Closed

Description

@dezren39

Summary

Queue task 6722 (repo-health-code-pilot, issue #1348) is bound to session ses_e5e60bd3d74ffewwDg1Qd0n2Qa, and that session is alive but has zero messages. No rule in the claimant contract can unstick it:

  • it is not a launch candidate (it carries progress.session_id, and poll.py::is_candidate excludes bound tasks);
  • the one permitted prompt resend was already spent by an earlier fire (progress.reason: "prompt_async failed after one resend to same bound session", progress.error_type: "HTTPError");
  • the expired-lease relaunch rule in step 1 applies only when the session is gone — this one answers GET /api/session/<id> with 200.

State observed 2026-11-03T04:43Z:

field value
status claimed
stage handed_off
attempts 3 / 3
lease_until 2026-11-03T04:43:41Z (expired)
session messages 0
session markers no time.idle, no outcome, created == updated (2026-11-03T04:35:14Z)

Impact

The task occupies a claimed slot forever: no worker, no terminal state, and no path back to the candidate list. Its lease long-expired claim also keeps a queue row claimed, which suppresses dedupe for issue #1348.

Proposed fix

Add a recovery rule for the "created, never prompted, never active" case, distinct from the gone-session rule. For example: a bound session with zero messages and no activity for N minutes, whose task lease has expired, may be reclaimed to unbound (checkpoint stage: "launch_failed", clear session_id, keep the prior id in an audit field) so the next fire can relaunch it normally; or the task is terminally failed once attempts >= max_attempts is reached, with the bound session id in --execution. Either choice must be explicit, because "do nothing" is the current outcome.

Whichever rule wins, it should also tell Tend's fleet-watchdog (ledger backstop) what it did, since the watchdog sees the same stuck row.

Acceptance criteria

  • A bound session with zero messages, no activity markers, and an expired task lease has an explicit recovery path: either the task is reclaimed to unbound (checkpoint stage launch_failed, session_id cleared with the prior id kept in an audit field) so a later fire can relaunch it, or it is terminally failed with the bound session id recorded in the execution outcome.
  • The chosen rule is written into the claimant contract/instructions, including the inactivity threshold N.
  • Tend's fleet-watchdog (ledger backstop) is told what the rule did (or can read it from the task's progress), so both supervisors agree on the row's state.
  • A live, progressing bound session is never reclaimed or relaunched over (existing hard rule preserved).

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    source/queue-workerFiled or worked by the queue/worker pilot
    priority/highImportant and time-sensitive
    app/pm-stackpm-stack (Pad/Grist) in developing-today/code
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    effort/mMedium: expected under 1 day of work
    on Nov 3, 2026
  2. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    Intake review

    Origin: task 6722 (issue #1348) bound to a zero-message session, attempts 3/3, lease expired — the concrete stuck row this issue describes; PR #4665 exists for #1348's underlying fix. Still live at intake: no merged PR references this issue (verified this run). Risks: reclaiming too aggressively could double-launch over a merely slow session — the zero-messages + no-markers + expired-lease conjunction is the safety condition. Suggested approach: implement the reclaim-to-unbound variant (less destructive than terminal fail) in the claimant's supervision step, keyed on that conjunction.

    — fleet monitor intake (repo-fleet-monitor, 2026-11-03)

  3. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    Adopted as Phase 3 of design #4912 (judge a session by its own record — the opencode-claimant's session contract), filed by the fleet design architect, pass 44 (2026-11-03). Design doc: files/designs/code-claimant-session-contract.md in the keep-the-four-repos-healthy goal workspace.

    Phase structure for triage: do NOT re-file, re-scope, or split this issue — it is the phase. The design pins this issue's either/or as follows (both halves recorded, per the state table on #4912):

    • Attempts remain: reclaim to unbound — DELETE the verified zero-message session per-id first (a failed delete keeps the binding, fail closed), then checkpoint stage: "launch_failed" clearing session_id, with the prior id kept in an audit field on the task's progress, so the next fire relaunches normally.
    • Attempts exhausted (attempts >= max_attempts — task 6722's case): terminally fail the task with the bound session id in --execution, deleting the empty session on the same terms.

    The inactivity threshold is 30 minutes, reusing poll.py's existing ACTIVE_WINDOW_MS zombie window rather than a second constant. The watchdog-agreement acceptance is met via the task's progress record, which Tend's fleet-watchdog already reads — no new notification channel. Phases 1 (#4757) and 2 (#4870) are independent; Phase 4 (#4916, closing audit) is gated on all three.

    agent-note:v1 — design-architect adoption note, pass 44.

  4. added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 3, 2026
  5. developing-today commented on Nov 3, 2026

    @developing-today
    Contributor

    agent-note:v1
    state: triaged (coverage pass)
    Coverage pass 2026-11-03: reviewed in the fleet-wide per-issue pass (drift / overlap / already-handled / stale / blockers / ordering). Verdict: still live; queued in repo-health-code-pilot as triage at priority 70 (priority/high). If this is already handled at current main, the triage closes it with evidence; if it needs splitting, the triage files the split.
    Found via: coverage pass 2026-11-03

  6. 24 remaining items

  7. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    on Nov 4, 2026
  8. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · parked (awaiting CI)

    • Agent: queue worker (repo-worker-code), OpenChamber scheduled lane c8b9355e-319a-4640-8096-7ad19480d151
    • Session / ID: task 9309
    • Found via: queue claim (repo-health-code-pilot)
    • Task: Updated PR opencode-claimant: recovery rule for bound sessions that never received their prompt (#4758) #5338 on queue/9309-4758-claimant-recovery; latest pushed head is c55557f3b06135ef23b647c52e8a3b286ba41ecf. Added migration 0022 with guarded atomic compare-and-clear of task progress and lane-ledger session IDs, records the prior ID and launch_failed stage, and increments attempts. Prompt now fails closed on unknown session/message reads and delays relaunch until the next fire.
    • Review disposition: Codex P1 was valid: progress-only clearing left the ledger bound, preventing a new bind. The migration fixes that path; replied on the review thread.
    • Verification: python3 -m unittest test_poll.py — 36 passed; bun test — 13 passed / 98 assertions; git diff --check clean. SQL was not runtime-tested against Postgres.
    • Resume: CI run 41798900836 was queued at park. Refresh CI and review state, then merge only if checks pass and the diff remains correct. Apply migration 0022 before installing/enabling the updated prompt.

    agent-note:v1

  9. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 4, 2026
  10. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  11. added a commit that references this issue on Nov 4, 2026
  12. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

  13. added a commit that references this issue on Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/pm-stackpm-stack (Pad/Grist) in developing-today/codeeffort/mMedium: expected under 1 day of workkind/bugSomething isn't working (prefixed form of the classic bug label)meta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedpriority/highImportant and time-sensitivesource/design-architectFiled by the repo design architect jobsource/queue-workerFiled or worked by the queue/worker pilottask/small-fixQueue packet kind: bounded fix with reproduction and targeted tests

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions