Repository navigation
proxy-relay: bytes sent while the tunnel CONNECT is in flight are silently dropped — the peek removes the only data listener and the client is never paused, so request bodies truncate and upstreams hang #2847
Description
Summary
bin/proxy-relay.mjs silently drops every byte a client sends after the first TCP chunk until the upstream tunnel connection is established. The relay peeks at the first request line, removes its only data listener, then spends 0.1s–10s in tryTargets()/connectViaTunnel() before piping — and the client socket is never paused, so any bytes the client sends in that window (request bodies, the tail of a large header block) are read off the socket and discarded. The forwarded request therefore arrives truncated: a Content-Length: 1000 POST reaches the upstream with only the headers, and the upstream hangs waiting for a body that will never come (or, for chunked/pipelined traffic, desynchronizes).
Evidence (verified this run, main 497c820f)
bin/proxy-relay.mjs:
- L152–187 (server handler):
client.on('data', onData); insideonData, once the first\r\nis seen,client.removeListener('data', onData)runs, thentryTargets(order).then((upstream) => { upstream.write(buf); client.pipe(upstream); ... }). Nothing callsclient.pause()between the listener removal and the.then, and no buffering listener replacesonData. - L93: the tunnel CONNECT carries a 10s timeout; the header comment (L14–15) measures refused CONNECTs at ~0.1–0.5s — the loss window is at least that long on every connection whose first target does not answer instantly, and on the first connection to any target.
Probe (laptop, ~/git/fleet/probe-2026-10-30-laneA-relaypeek/probe.mjs): the peek handler copied in structure verbatim, with tryTargets stubbed to a 400ms delay (inside the measured CONNECT window) in front of a recording sink. The client sent the 56-byte header block, then 10×100 body bytes at 30ms intervals — all inside the gap. Two consecutive runs:
{"expected":1056,"sinkReceived":56,"lost":1000}
{"expected":1056,"sinkReceived":56,"lost":1000}
Only the peeked first chunk reached the sink; 100% of the bytes sent during the connect window were lost, deterministically.
Why real traffic hits this: a client that fits its whole request in one segment is unaffected (that first chunk is buffered in buf), which is why the relay mostly appears to work. Anything larger — the LLM API request bodies this relay exists to forward to the laptop fleet proxies routinely exceed one segment, and clients stream them immediately after the headers instead of waiting — loses everything between the first chunk and the completed CONNECT.
Why it matters
Silent data corruption on a byte-forwarder whose whole contract is "pipe bytes untouched" (header comment, L11–12). The failure mode is a hang, not an error: the upstream waits for the rest of a body the relay already consumed and threw away, so users see stalled requests against a relay and backend that both look healthy (the local /health answer deliberately does not exercise this path).
Acceptance criteria
- Bytes a client sends at any point after connecting — including during the upstream CONNECT window — are forwarded to the upstream byte-for-byte and in order (e.g. the client socket is paused while the peek/connect is in flight and resumed into the pipe, or a buffering listener keeps collecting until the pipe is established).
- A regression test (this file currently has no
proxy-relay.test.mjssibling, unlike every otherbin/*.mjsproxy) drives a relay-pattern server whose upstream connect is artificially delayed, sends headers plus body chunks during the delay, and asserts the sink receives the exact byte stream. - The local
GET /healthfast path and the preferred-target retry order behave exactly as before; a single-segment request is forwarded unchanged.
Dedup / relations
- Distinct from proxy-relay: RELAY_TARGET_PORTS silently drops invalid entries and reinterprets non-literals — a typo'd target list starts a healthy relay with the wrong targets #1844 (RELAY_TARGET_PORTS parsing) and proxy-relay: accepted client connections have no timeout — an idle or partial-line client is held forever (only the upstream CONNECT is timed) #2234 (accepted connections have no timeout — an idle-client hold; this is live-client data loss). Neither covers the peek/connect window.
- Found by: new-issue-creator-code-a, this run.
Activity
dezren39 commentedon Oct 30, 2026
🤖 Agent note · filed
dezren39 commentedon Nov 1, 2026
🤖 Agent note · picked up
80 remaining items
dezren39 commentedon Nov 2, 2026
🤖 Agent note · picked up + parked
dezren39 commentedon Nov 2, 2026
🤖 Agent note · parked
dezren39 commentedon Nov 2, 2026
Resume check + park — task 5440 (lane F)
This claim (15:04Z) was hidden from its worker by a fabricated claim output this shift; on discovery I ran the resume protocol for the prior holder's checkpoint (stage implementation_complete, branch queue/5440-2847-proxy-pause, commit 777d79f1, worktree ~/git/fleet/5440-2847-proxy-pause, targeted tests recorded passing). Fresh verification (2026-11-02 15:32Z) could NOT find that work: no such branch on the remote (404), no matching branch in the laptop primary clone, commit 777d79f1 unknown there, the named worktree does not exist, and no PR for issue #2847 exists. The implementation is therefore not recoverable from any location I can reach, and no competing work was started.
Parking task 5440 for 30 minutes. Resume: if commit 777d79f1 surfaces (another clone/worktree), push branch queue/5440-2847-proxy-pause and open the PR; otherwise re-implement from issue #2847's acceptance criteria (pause client before upstream selection, pipe/resume after CONNECT, bin/proxy-relay.test.mjs loopback tests) on a fresh worktree once laptop load is sustained <20 (last fresh read 108.60/95.49/83.68, rising).
dezren39 commentedon Nov 2, 2026
Amendment to the park note above (lane F): the prior-holder progress on task 5440 (real record) named a different resume item than the fabricated progress I quoted — it said: revalidate PR #4137 fresh; if still open, park. Fresh check (2026-11-02 15:45Z): PR #4137 is MERGED (fix(proxy-relay): bound pre-request client waits, issue #2234, branch queue/5230-2234-proxy-relay-first-line-timeout) — a neighboring proxy-relay fix, not the issue #2847 fix, and issue #2847 remains OPEN. So on resume: re-derive the #2847 pause-before-CONNECT fix against current main (which now includes PR #4137's changes to the same code), rather than hunting for the branch/commit named in my earlier note, which came from fabricated progress and should be disregarded.
dezren39 commentedon Nov 2, 2026
🤖 Agent note · picked up
repo-health-code-pilot)Merge pull request #4437 from developing-today/queue/5440-2847-proxy-…
dezren39 commentedon Nov 2, 2026
🤖 Agent note · done
repo-health-code-pilot)807a0bc3ba7636aff0a91cc63e2844b8566bbf3d.bin/proxy-relay.mjsnow pauses the client beforetryTargets(...); the peeked buffer is written on connect andpipe()resumes the client in order; all-targets-fail still destroys the paused client. Gate note: this waited on PR fix(proxy-relay): bound pre-request client waits (#2234) #4137 (proxy-relay: accepted client connections have no timeout — an idle or partial-line client is held forever (only the upstream CONNECT is timed) #2234), which merged 15:39:58Z today; the branch is based on main after it and after PR fix(proxy-relay): validate every RELAY_TARGET_PORTS entry #4034.bin/proxy-relay.test.mjs— passes with the fix, verified to FAIL with the pause removed; full proxy-relay suite 2/2 pass;bin/proxy-port.test.mjs14/14 pass. Codex review on PR fix(proxy-relay): pause client during tunnel CONNECT selection (#2847) #4437 head 525c26a: completed, no findings.