Skip to content

design-code-proxy: backend dispatch has no timeout — a backend that accepts and never responds hangs the request forever and the fallback never fires #2097

Closed

Description

@dezren39

Summary

bin/design-code-proxy.mjs dispatch has no timeout on its backend requests: if a backend accepts the connection (or connects and then stalls) and never answers, the client request hangs forever and the proxy's own primary→fallback machinery never fires — fallback only triggers on a socket error, an HTTP 429/5xx response, or an in-body quota error, and a silent backend produces none of those. Each hung request also holds one socket from the shared keep-alive pool (maxSockets: 32 per scheme) indefinitely, so a wedged backend can gradually exhaust the pool for the healthy backend too.

Evidence (main bd880194, file sha256 27e9fc8c…)

  • bin/design-code-proxy.mjs — tryTarget (in dispatchWithFallback) builds the backend call with http.request({ hostname: '127.0.0.1', port: target.port, path: '/v1/chat/completions', agent: httpAgent, … }) and registers only localReq.on('error', …) → onFail. There is no localReq.setTimeout(...), no timeout option, and no abort anywhere in the dispatch path. The only AbortSignal.timeout in the file is 2500 ms on the background /v1/models catalog probes (:95).
  • The AMR upstream forward in the same file (clientReq = https.request(...), same handler) likewise sets no timeout; its error handler falls back to local dispatch, which a hanging upstream also never triggers.
  • The fallback contract is otherwise proven working: a primary that answers 500 falls back immediately (see probe control below).

Probe (laptop, 2026-10-28, shipped file unmodified)

Probe dir: ~/git/fleet/probe-2026-10-28-laneB-designtimeout/ (laptop) — two stub backends (primary switchable 500/hang, fallback healthy) + the shipped proxy on alternate ports (ANTIGRAVITY_LS_PROXY_PORT/CODEX_PROXY_PORT/DESIGN_CODE_PROXY_PORT env):

  • Control, healthy primary (gpt-6-luna → codex stub): 200 in 94 ms.
  • Control, primary answers 500 (design-code-pro → antigravity stub 500, codex stub healthy): 200 in 31 ms via fallback — the fallback machinery works when failure is signalled.
  • Test, primary accepts and never responds (design-code-pro, primary in hang mode): no response within 8000 ms (client-side AbortSignal timeout fired); the healthy fallback was never tried. The proxy itself stayed alive (GET /v1/models → 200 immediately after), so this is a per-request permanent wedge, not a crash.

Why it matters

dispatchWithFallback exists precisely so one bad backend doesn't take down design-code requests — the file already handles backend response errors, 429/5xx, and in-body quota errors (#681 fixed a sibling "hangs forever instead of falling back" bug in this same function). A backend that wedges silently (deadlocked language server, a proxy sibling stuck on its own upstream) is the remaining failure mode, and it is worse than a clean error: the client waits indefinitely and pooled sockets accumulate.

Acceptance criteria

  • A backend that connects but does not respond within a bounded time (triage pins the value; it should be generous enough for slow streaming first-byte, e.g. a time-to-first-response / socket-inactivity timeout rather than a total-stream deadline) causes onFail → fallback, and if both targets time out, a clean 502 — never an indefinite hang.
  • The AMR upstream forward (https.request path) gets the same treatment: a silent upstream falls back to local dispatch instead of hanging.
  • Streaming responses that are actively flowing are not killed by the timeout (inactivity-based, or reset on data).
  • Tests in bin/design-code-proxy.test.mjs style: hanging-primary → fallback serves; both-hang → 502 within a bounded time; slow-but-alive backend still succeeds.
  • Existing behaviours unchanged: 429/5xx fallback, in-body quota fallback, model rewriting on stream and non-stream paths.

Activity

  1. developing-today commented on Oct 29, 2026

    @developing-today
    Contributor
  2. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    priority/normalOrdinary priority (the default)
    queue/queuedHas a queued task in the Supabase agent queue
    source/creatorFiled by the new-issue-creator job
    task/triageQueue packet kind: verify, reproduce, and route this item
    effort/sSmall: expected under 2 hours of work
    on Oct 29, 2026
  3. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · filed

    • Agent: new-issue-creator-code-b
    • Session / ID: cron new-issue-creator-code-b run 2026-10-28 22:20 CDT
    • Found via: filed this run
    • Task: Triage design-code-proxy's missing backend timeout (bin/design-code-proxy.mjs tryTarget + AMR forward): pin an inactivity/first-response timeout value, wire it to onFail so the fallback fires, add hanging-backend tests.
    • Expectation: effort/s; done = a silent backend falls back (or 502 when both hang) within a bounded time, actively-streaming responses are not killed, existing 429/5xx/quota fallbacks unchanged.
    • Links: evidence probe ~/git/fleet/probe-2026-10-28-laneB-designtimeout/ (laptop, main bd88019, file sha256 27e9fc8c); sibling fixed case design-code-proxy: quota error in a 200 JSON body hangs the request forever — writeHead precedes the in-body check, so fallback can never fire #681 (same function, different hang cause)
  4. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 1, 2026
  5. 68 remaining items

  6. added
    status/parkedDeliberately paused with a recorded resume condition
    queue/claimedQueue worker lifecycle state
    and removed
    queue/claimedQueue worker lifecycle state
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 2, 2026
  7. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  8. dezren39 commented on Nov 2, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/design-code-proxybin/design-code-proxy.mjs fleet proxyeffort/sSmall: expected under 2 hours of workkind/bugSomething isn't working (prefixed form of the classic bug label)priority/normalOrdinary priority (the default)source/creatorFiled by the new-issue-creator jobtask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions