Skip to content

fleet step 3: proxy domains — hello/capability frame, PATH shim regeneration, fleet init #5431

Closed

Description

@developing-today

Part of epic #5412. Design of record: #5425 (§4, §5, §7 step 3 — "Proxy"). Predecessor: step 2 (queue pilot). Gate for step 4: negotiation demonstrated across protocol majors, shim regeneration proven against a changing services map, and the SSH proof period completed before the ProxyCommand re-point.

Scope

  • fleet-proxy lands both roles in one program: fleet proxy serve — the broker (unix socket at today's path, per-port connection pools for 4799/4800/4801, auto-start on missing socket, 1-hour idle exit retained) — and fleet proxy connect / fleet init — the client, whose autostart is an idempotent no-op when the broker already answers.
  • Protocol (D8): today's length-prefixed JSON framing is kept — no redesign. A versioned hello/capability frame becomes the first frame on every pooled connection; its services map is the routing table. The broker accepts protocol majors N and N−1; within a major, fields are additive-only. The framing and hello live in fleet-proxy-proto and nowhere else.
  • PATH shims are regenerated from the negotiated services map on every connect/init. No shim survives from an older negotiation, so what is on PATH always matches what the broker actually routes — the anti-drift mechanism behind "repo-queue on PATH the moment you're connected." Per §8's scope-gravity risk, the shim layer advertising external tools (id, omnibin-resolved programs) is advertising only — it must not become ownership of them.
  • Serve side persists nothing: the routing table is config, pooled tunnels are memory, and keepalive must not reset the idle exit (§5).
  • Takeover order: fleet proxy serve takes over the sandbox socket path (the Python broker keeps its socket until the flip; the flip is a restart with the same path) → connect/init replace the client autostart habits → fleet-connect ships to codespaces/CI from here.
  • SSH retires last. laptop-proxy-connect.py (SSH's ProxyCommand) re-points to fleet proxy connect's stdio mode only after SSH-through-fleet has been proven over days, not minutes. The laptop-agent daemon itself stays Python in v1 (D5); fleet manages its lifecycle/health only.

Gate

  • Hello/capability negotiation demonstrated between a current build and an N−1 build in both directions.
  • Shim regeneration shown to add and remove tools as the negotiated services map changes.
  • proxy serve + connect round-trip green on the real socket path; idle-exit behavior unchanged.
  • SSH-through-fleet proof period (days) completed and recorded before the ProxyCommand re-point, which is itself a separate reversible flip.

Acceptance criteria

  • proxy domains (hello/capability frame, PATH shim regeneration, fleet init) work as described.
  • No regression: existing tests/checks for the touched area remain green, and the change lands via PR referencing this issue.
  • Evidence (PR link, test/CI output, or verification note) is posted on this issue before it is closed.

Activity

  1. dezren39 commented on Nov 4, 2026

    @dezren39
    Member

    Intake review

    🤖 Agent note · state: filed

    • Agent: fleet monitor
    • Session / ID: repo-fleet-monitor run 2026-11-04 06:12 CST
    • Found via: monitor reconciliation
    • Task: Intake enrichment (protocol §5): verified still live (no merged PR or closed duplicate found naming this fix in a bounded check this run), appended acceptance criteria, normalized labels.
    • Expectation: as labelled (effort class on the issue); done = acceptance criteria met with evidence linked here.
    • Links: this issue

    Related: epic #5412 and steps #5429-#5433 form one set; #5413 is a sibling absorption task.
    Notes: Newly filed this morning with no priority label and no acceptance-criteria section; both now supplied.
    Risks: Sequencing: steps are gated in order (notably #5433 on the #5350 step 6 cutover); starting out of order risks rework.
    Suggested approach: Triage through the queue in dependency order, one step per task, verifying each step's acceptance evidence before closing.

  2. added
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    on Nov 4, 2026
  3. added a commit that references this issue on Nov 4, 2026
  4. dezren39 commented on Nov 4, 2026

    @dezren39
    Member

    Step 3 landed — the proxy domain (PR #5515, merged 5521f3f11)

    Sandbox ~/tools/fleet runs the merged head rev 0377b9a11; the session broker on the sandbox IS fleet proxy serve as of this comment.

    Gates

    • Hello negotiation, N and N−1, both directions. N−1 is proto 0 — the legacy dialect the Python broker and clients speak (the window arithmetic already admitted it). Fleet broker → legacy client: ~/tools/laptop-agent.py health returns the real daemon health through fleet serve on the production socket. Legacy broker → fleet client: before the flip, fleet proxy connect --request GET /health against the still-running Python broker negotiated the legacy fallback and round-tripped (200, real health body). Out-of-window hello → proto_too_old frame → client exit 69 (proto-crate unit tests).
    • Shim regeneration from the negotiated map. Smoke proves materialization and stale-shim removal when the map shrinks; live, ~/.local/state/fleet/shims holds laptop-agent + repo-queue, regenerated at the flip — links re-pointed from the proof binary to ~/tools/fleet when the source binary changed, untouched when nothing changed.
    • fleet init no-op. Live: fleet: proxy already running (proto 1); shims current, and exactly one broker process before and after (pid-verified). Smoke asserts the same hermetically.
    • Real broker round-trip. repo-queue health (the production queue, port 4801), laptop-agent.py health (4799), and secrets-broker /health (4800, token file named per request — no secret fetched) all green through fleet serve.
    • Real SSH session through fleet's proxy. One-off ssh -o ProxyCommand="$HOME/tools/fleet proxy connect --stdio --host %h --port %p" … 'echo SSH_THROUGH_FLEET_OK; hostname' → OK / amd, ~2 s.
    • Idle exit. Hermetic smoke: an 8-second-idle broker exits on schedule while its keepalive loop runs (keepalive ≠ use). The production broker keeps the 3600 s default; its keepalive serves the same /health cadence.
    • Per-variant smoke: 55/55 (incl. D7 size gate: connect 751,168 B dynamic / 902,560 B static). Workspace tests 42/42, clippy clean.

    Retirement gate — exact state

    The note (§4 takeover order, §7): "SSH retires last: laptop-proxy-connect.py (SSH's ProxyCommand) re-points to fleet proxy connect stdio mode only after SSH-through-fleet proven over days, not minutes." Not taken. laptop-proxy-connect.py and ~/tools/laptop-ssh.sh are untouched and remain the default SSH route; the proof period started 2026-11-04. Rollback inventory intact: ~/tools/fleet.step2-bak (previous binary), the Python broker (still what laptop-agent.py auto-starts if the socket dies, and init-ensure's fallback), the fleet-brokers standalone package.

    ~/tools/init-ensure.sh now prefers fleet proxy serve with the Python spawn as fallback when no fleet binary exists, and its running-gate accepts either broker process (hatch-personal 6d6e98c3c).

    What the note (and step 1) got wrong about the live system

    1. §4 framing: "length-prefixed (4-byte)". The live wire is an 8-byte big-endian prefix (struct.pack(">Q") — Python broker, Python client, and step 2's fleet-queue all agree). Step 1's fleet-proxy-proto implemented 4-byte, which silently broke fleet-secrets' broker source — it could never have completed a round-trip with the live broker; the failure was masked by its bws fallback. Separately, fleet-config's secrets-token default pointed at .config/secrets-broker/token, a path nothing on the live system has; the live file is ~/tools/secrets-broker-token. Both fixed here; the proto crate carries the erratum in its docs.
    2. fleet init existed only on paper. Step 2's dispatch rework dropped the top-level verb (unrecognized subcommand 'init'); §8 and this issue both name it as the autostart entry. Restored, laddered on the proxy-serve capability like the other verbs.
    3. Implementation finding (not a note error): splice(2) stalls here. std::io::copy lowers to splice on Linux; through fleet proxy connect --stdio, the SSH handshake froze mid-stream (stall point varied by run — banner, KEX, ECDH) while laptop-proxy-connect.py's recv/sendall ran the identical path instantly. Traced live with strace. The relay now uses explicit read/write pumps — the Python's syscall shape — and the SSH proof above runs on it. Any future stdio/splice use in this sandbox should treat splice as suspect.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/infraInfrastructure, deployment, and operationseffort/mMedium: expected under 1 day of workkind/enhancementmeta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedpriority/normalOrdinary priority (the default)status/needs-triageNot yet triaged; needs a first look

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions