Skip to content

Shift batching, creator-style: lane handoff snapshots + supervision mode for queue lanes #2140

Open

Description

@dezren39

Queue shifts already bundle up to 10 tasks; their cost (6–13M input/run at check-in #5; the 9-lane shift family was 52% of all cron spend) is per-start state re-derivation plus in-shift supervision context — the same overhead class the creator batching change attacked (3.93M → ~1.4M input per filed issue).

Do:

  1. Per-lane handoff snapshots: persist checked state + task checkpoints between shifts so a shift starts from the snapshot instead of re-reading the full instruction/queue/GitHub/log state (fleet-watchdog's SQL-first rewrite is the template).
  2. Supervision mode: as dispatcher session↔task binding lands (the binding gap tracked on the queue pilot), lanes shift from executing tasks themselves to batch-supervising dispatched OpenCode sessions — target 15–20 supervised sessions per shift.
  3. Evaluate 40m → 2h cadence once supervision mode is real; report per-queue tokens per completed task before/after in the daily check-in.

Activity

  1. developing-today commented on Oct 29, 2026

    @developing-today
    Contributor
  2. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    TOP PRIORITY (Drew, 2026-10-29 01:23 CDT): Batch 3 fleet consolidation — pick this up in the next runs, ahead of routine work. Being enqueued to repo-health-code-pilot at top priority now.

  3. added
    queue/queuedHas a queued task in the Supabase agent queue
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Oct 29, 2026
  4. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  5. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

    Triage verdict — premise CONFIRMED, design decided, Part 1 routed

    Premise check: verified from the inside — this triage was executed by a Lane B shift that itself re-read the full worker instruction, re-derived queue health (including one 12s health timeout + retry), and re-validated every task's GitHub state from zero, exactly the per-start overhead this issue names. The scale figures stand as filed (9-lane family = 52% of cron spend at check-in #5; 6–13M input/run; creator batching precedent 3.93M → ~1.4M per filed issue). One refinement: shifts do not start from nothing — checkpoints and progress survive in the queue itself. What is missing is a lane-level digest of everything around the tasks (infra state, in-flight PR/CI pointers, dead ends already discovered), which is what the snapshot below adds. It complements task checkpoints; it does not replace them.

    Design

    Part 1 — lane handoff snapshots (landable now)

    • Write: at shift exit (after the signal refresh, before lock release) the lane overwrites hidden_files/lane-handoff-<queue>-<lane>.json:
      {generated_at, lane, queue_health:{queued, claimed, parked, blocked}, worked:[{task, issue, outcome, pr, links}], in_flight:[{task, issue, pr, ci_state, resume_condition}], infra_notes:[{note, first_seen}], leads:[task ids worth claiming next and why]}
    • Read: the next shift of the same lane reads it right after the Step 0 gate. Leads, not truth: the snapshot replaces search (what exists, where things stand, what was already ruled out) and never verification — claims, CI state, and labels are still checked fresh before acting, and anything contradicted by a fresh check loses to the fresh check. Entries older than ~2 cadences are context only.
    • Why this shape: it is the fleet's proven digest pattern (watchdog's live-state digest, the monitor rewrite in Slim repo-fleet-monitor: snapshot/SQL-first state fetch + 2h cadence (target ≤3M/run) #2134) applied to the lane boundary; one small file per lane, overwritten, no log archaeology, no new store.
    • Success test: daily check-in reports per-lane input/run for two weeks post-landing vs the 6–13M range; success = median down with tasks/shift held or rising.

    Part 2 — supervision mode (designed, GATED — not routed for implementation)

    • Gate (both required): (a) the session conductor (Implement the session conductor: merge openchamber-dispatcher + openchamber-babysitter into openchamber-conductor #2153) is cut over, and (b) session↔task binding is recorded on queue tasks (a claimed task's progress names its OpenCode session id). Until both hold, lanes keep executing directly; a half-bound supervision mode would recreate the dispatcher/babysitter split this batch just merged.
    • Mode, once gated in: a shift batch-supervises 15–20 dispatched sessions (checkpoint reviews, stall nudges through the conductor's escalation ladder, hand-backs with reasons) and executes directly only what no session holds.
    • Cadence: the 40m → 2h evaluation happens only after the gate, and a cadence change is a rate change — it needs Drew's explicit word at that time, with before/after tokens-per-completed-task from the daily check-in as the evidence pack. Nothing in this triage pre-approves it.

    Routing

    Part 1 implementation filed as #2158 (snapshot steps in all lane shift bodies, one evidenced handoff, two-week measurement; Part 2 gate state recorded, not started) and enqueued as queue task 3183, flagged live-fleet-change. Role note: lanes stay Do in the #2131 map in both parts; supervision changes how Do works, not which role owns it.

  6. dezren39 commented on Oct 30, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · Part 2 gate state (recorded per #2158 acceptance, queue task 3812, 2026-10-30)

    Gate NOT satisfied on either count — no supervision mode started, no cadence change made.

    • (a) Conductor cut over? No. openchamber-dispatcher and openchamber-babysitter are both still enabled: true in the live cron roster (verified 2026-10-30); the conductor cutover for Implement the session conductor: merge openchamber-dispatcher + openchamber-babysitter into openchamber-conductor #2153 is queued as task 3806 and has not landed.
    • (b) Session↔task binding present? No. Queue task progress records stage/artifact/next only — no OpenCode session id is recorded on a claimed task (observed across this shift's claims: tasks 3311, 3557, 3415, 3559, 3647, 3696, 3733, 3797, 3808, 3812).

    Part 1 (per-lane handoff snapshots) landed today under #2158: snapshot write + read steps are in both shared lane instruction files, and the two-week input/run measurement clock starts 2026-10-30. When both gate conditions above flip, file the Part 2 activation as a fresh follow-on from this issue — the 40m → 2h cadence evaluation additionally needs Drew's explicit word even then.

  7. 17 remaining items

  8. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

    • Agent: fleet monitor (repo-fleet-monitor)
    • Session / ID: task 7304
    • Found via: monitor reconciliation
    • Task: claim-evaporated repair — the queue claim on task 7304 ended without complete/park/block, its lease expired, and the task returned to queued unclaimed while this issue still carried claimed labels. Labels reset to the truth: queue/queued + status/needs-triage. Checkpoint stage at evaporation: handed_off.
    • Expectation: the next lane to claim task 7304 posts a fresh pickup note.
  9. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 4, 2026
  10. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

    • Agent: queue worker (repo-worker-code), lane code-j
    • Session / ID: task 7304
    • Found via: queue claim
    • Task: Triage Shift batching, creator-style: lane handoff snapshots + supervision mode for queue lanes #2140: Shift batching, creator-style: lane handoff snapshots + supervision mode for que handed off to the opencode execution lane per task 7304 lane assignment (assigned_lane=opencode, session_id None)
    • Expectation: claimant launches the bound opencode session; that session drives task 7304 to its terminal state
  11. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  12. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/claimedQueue worker lifecycle state
    on Nov 4, 2026
  13. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

priority/highImportant and time-sensitivequeue/parkedQueue task is parked (waiting on CI, time, or an event)source/creatorFiled by the new-issue-creator jobstatus/parkedDeliberately paused with a recorded resume conditiontask/triageQueue packet kind: verify, reproduce, and route this itemwork/assignedQueue worker lifecycle state

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions