Repository navigation
pm-dispatch(queue): ensureWorker caches its first failure forever — one transient Supabase 500 permanently wedges every session launch until restart #2663
Description
Problem
In pm-dispatch's queue client (pm-stack/dispatch/src/queue.ts, main 732ca26), QueueClient.ensureWorker() memoizes its registration promise permanently — including when it rejects:
private workerEnsured: Promise<void> | null = null; // queue.ts:132
ensureWorker(): Promise<void> {
this.workerEnsured ??= (async () => {
const res = await fetch(`${this.baseUrl}/rest/v1/workers`, { method: "POST", /* … */ });
if (!res.ok) {
const text = await res.text();
throw new QueueError(`worker registration failed (HTTP ${res.status}): …`);
}
})();
return this.workerEnsured; // queue.ts:137-158
}The ??= assignment happens once, with the in-flight promise. If that first attempt fails — one transient Supabase/PostgREST 500, one network blip at service start — the rejected promise is cached for the life of the process, and every later ensureWorker() call rethrows the original error without making a new request. Since launchSession (src/server.ts:130) awaits ensureWorker() before the claim PATCH, every subsequent POST /v1/sessions launch fails at that step until the service is restarted — even though the worker-row upsert is idempotent (resolution=ignore-duplicates) and a retry would succeed.
Evidence (verified this run)
Laptop probe (bun, importing the real queue.ts from the laptop primary at main 732ca26, md5-identical to the sandbox clone; probe at ~/git/fleet/probe-2026-10-30-laneB-dispatch/ on the laptop), with global fetch stubbed to return HTTP 500 on the first call and HTTP 201 thereafter:
B ensureWorker: fetchCalls= 1 firstErr= worker registration failed (HTTP 500): boom secondErr= worker registration failed (HTTP 500): boom
One fetch was ever made; the second call rethrew the cached rejection instead of retrying.
Why it matters
pm-dispatch is a long-lived service. A single transient failure during its first launch permanently wedges all session launches (the service's primary endpoint) behind an error message that points at a failure that is no longer happening — the worst kind of stale error to debug. The enqueue-only /v1/tasks path keeps working, so the service looks half-alive.
Acceptance criteria
- A failed
ensureWorker()attempt is not cached: the next call retries the registration request (e.g. clearworkerEnsuredon rejection, or only memoize after success). - A successful registration is still memoized (no per-launch POST).
- A regression test with a fail-then-succeed stubbed transport pins both behaviors (fetch call counts: 2 across fail+retry, 1 across two successes).
- Existing dispatch tests still pass.
Activity
dezren39 commentedon Oct 30, 2026
🤖 Agent note · filed
dezren39 commentedon Oct 30, 2026
agent-note:v1 — design-architect adoption (pass 19, 2026-10-30): this issue is Phase 3 (retryable init) of design #2668, "pm-dispatch launch pipeline, staged". Design doc:
files/designs/code-pm-dispatch-launch-pipeline.mdin the keep-the-four-repos-healthy goal workspace.Recorded decision for triage (do not re-file, re-scope, or split this issue; record the phase structure instead): the memo caches success only —
ensureWorker()clearsworkerEnsuredwhen the promise rejects, so the next launch retries the idempotent (ignore-duplicates) registration while concurrent callers still share the one in-flight promise. This is thefetchGithubJwkssuccess-only-cache pattern already inauth.ts. No backoff loop or background timer: call-driven retry is the recorded design. Verified at design time:workerEnsuredis the only??=promise memo inpm-stack/dispatch/src/.This issue already carries a queued triage packet; no new packet was enqueued by the architect.
dezren39 commentedon Nov 1, 2026
🤖 Agent note · done