design: remote desktop through the fleet brokers (RustDesk transport, secrets, pre-warming, virtual desktops) #5694

Epic: #5412 · Parent design: #5343 · Related: #5509 (dedicated RustDesk SSH forward)

Context

Remote desktop to the laptop today works, but not through the fleet:

• RustDesk (physical desktop) goes over a dedicated raw SSH forward
  (~/tools/rustdesk-gl/forward.sh: sandbox 127.0.0.1:21118 → laptop
  127.0.0.1:21118, its own non-multiplexed ssh -N -L, isolated after
  RustDesk: login succeeds but no video — PipeWire ScreenCast negotiation stalls under Hyprland (amd-6) #5509 root-caused freezes to contention on the shared laptop-ssh
  ControlMaster). It works and sessions run indefinitely, but it is a
  second, SSH-shaped transport beside the fleet brokers, with no pooling,
  no preconnect, and no place in the fleet services map.
• Chrome Remote Desktop (the amd-crd container on the laptop) gives
  a virtual desktop that shares the laptop's filesystem and is decoupled
  from the physical desktop login — which matters because laptop desktop
  login state has been a recurring blocker for the physical session. It
  is Google-brokered and is a separate world from the fleet.
• Fleet proxy is a TCP CONNECT proxy: tunnel.rs opens raw tunnels
  through the runtime egress proxy, serve.rs pools them per port
  (default 50) and pre-warms the laptop-agent pool at startup — but the
  broker protocol only carries HTTP request/response JSON for services
  (ports 4799/4800/4801 today), never a raw long-lived byte stream.
• Sandbox constraint: the sandbox reaches the tailnet only via the
  runtime proxy CONNECT — TCP only, no UDP. RustDesk's rendezvous /
  hole-punch machinery uses UDP heavily, so any fleet design must work
  in a TCP-only, relay/direct shape, not the hole-punch shape.

This issue answers, from RustDesk's actual docs/source: where the
secrets live, whether the broker can relay (and pre-warm) a desktop
transport without holding any of them, what concurrent clients mean on
a RustDesk host, and what per-connection virtual desktops would take.

Upstream sources used throughout:

• RustDesk self-host docs — https://github.com/rustdesk/doc.rustdesk.com/blob/HEAD/content/self-host/_index.en.md
• RustDesk advanced settings (direct-server, direct-access-port) — https://RustDesk.Com/docs/en/self-host/client-configuration/advanced-settings/
• RustDesk client/host source — https://github.com/rustdesk/rustdesk/blob/master/src/client.rs, https://github.com/rustdesk/rustdesk/blob/master/src/server.rs, https://github.com/rustdesk/rustdesk/blob/master/src/server/connection.rs, https://github.com/rustdesk/rustdesk/blob/master/src/rendezvous_mediator.rs
• RustDesk server source — https://github.com/rustdesk/rustdesk-server/blob/master/src/rendezvous_server.rs (hbbs), https://github.com/rustdesk/rustdesk-server/blob/master/src/relay_server.rs (hbbr)

Fleet sources are cited as pkgs/fleet/…:line in this repo.

Findings (a) — Secrets, in three separate layers

RustDesk conflates easily; the layers are in fact cleanly separated:

1. Rendezvous / ID registration (who is who). Every host registers
with the ID server hbbs (TCP+UDP 21116): RegisterPeer announces the
peer ID + address; RegisterPk registers the peer's uuid and public
key, and hbbs stores id → (uuid, pk, ip) and hands the pk back out
signed (signed_id_pk) inside punch-hole / relay responses
(rustdesk-server rendezvous_server.rs:396,412,488,596,709).
What a rendezvous server holds: peer IDs, public keys, IPs — and its
own server signing key pair (the key clients pin in Settings →
Network). No session password is ever registered.

2. Transport establishment (how bytes get there). hbbs tries to
hole-punch A↔B (TCP 21116, UDP 21116, plus host-side TCP/UDP/KCP/WebRTC
punches). On failure both sides dial the relay hbbr (TCP 21117) with
RequestRelay { uuid, licence_key }; the first arrival waits in a
PEERS map (30 s pairing window) and the second with the same uuid
is spliced to it, set_raw() on both streams, opaque forwarding from
then on (rustdesk-server relay_server.rs:461–492).
What a relay needs: the pairing uuid (a per-connection nonce issued
via rendezvous), and optionally the server licence_key if hbbr
was started with -k — that key gates relay use, it is not the
session password. A relay needs no peer password, no peer private
key, and does not even need the peer's public key (the pk travels
client→client via the signed rendezvous response, not via hbbr).

3. Session authentication (may this client drive the desktop).
This happens host-side, after transport, inside the connection —
in the controlled host's Connection (rustdesk src/server/connection.rs):

• On open the host immediately sends Hash { salt, challenge }, the
  challenge freshly minted per connection
  (Connection::start, salt from the permanent-password config,
  challenge: Config::get_auto_password(6); on_open sends the Hash,
  connection.rs:545–549, 1510–1531).
• The client answers in LoginRequest.password with the proof
  SHA256( SHA256(password + salt) + challenge )
  (client: handle_hash / handle_login_from_ui, client.rs:4647ff,
  4872ff; host: verify_h1 / validate_password_plain,
  connection.rs:2523–2560). The password itself is never sent, in
  cleartext or otherwise.
• Which password is accepted is policy, not protocol:
  verification-method = temporary (the rotating one-time password
  shown on the host) / permanent / both (default); crossed with
  approve-mode = password / click (person at the host approves) /
  both (default password-click) — hbb_common
password_security.rs:42–90. Optional TOTP 2FA on the host gates
  authorization after the password proof.
• On the rendezvous/relay path the proof additionally travels inside
  the encrypted session: the client verifies the hbbs-signed peer
  pk (decode_id_pk against the pinned server key), generates a fresh
  symmetric session key, seals it to the host's ephemeral box pk
  (create_symmetric_key_msg), and the stream runs NaCl secretbox
  from then on (secure_connection, client.rs:1655ff; host side
  identity_handshake, server.rs:262ff). Password verification is
  therefore end-to-end: rendezvous sees identity, relay sees
  ciphertext, host alone checks the proof.

Direct answer: yes — a TCP transport to a RustDesk host can be
opened and its bytes relayed end-to-end without anyone in the middle
holding, seeing, or being able to check the session password. That
is exactly hbbr's design, and a fleet CONNECT splice inherits it:
the broker needs a target host:port and nothing else. No token file,
no password in broker config, no password in broker logs — the broker
cannot parse the stream it splices.

Two caveats that must be designed around, both from source:

• The direct-IP path is NOT the encrypted path. When the client is
  given an IP literal it dials check_port(peer, RELAY_PORT + 1) =
  21118 directly (client.rs:440–455), has no signed_id_pk, and
  secure_connection falls through: it sends an empty message and
  proceeds unencrypted. Symmetrically the host's direct listener
  calls create_tcp_connection(…, secure = false, …), so
  identity_handshake is a no-op (rendezvous_mediator.rs
direct_server, ~1354–1424; server.rs:213–260). The hash/challenge
  proof still never exposes the password itself, but on this path the
  proof (and then the whole desktop stream) has no RustDesk-layer
  confidentiality or host authentication — and the proof is a fast
  double-SHA256 with no slow KDF, so a captured proof is
  offline-brute-forceable against a weak permanent password.
  Consequences: (i) on the direct path, transport confidentiality must
  come from our tunnel (the runtime-proxy CONNECT path / tailnet),
  never from exposing 21118 raw; (ii) the laptop's permanent password
  must be long/random, since it is the only thing standing behind the
  proof; (iii) this is precisely today's forward.sh shape too — SSH
  supplies the encryption RustDesk direct mode lacks.
• Fail-open downgrade exists on the relay path too: if the signed
  pk is missing/invalid, the client logs and falls back to a
  non-secure connection rather than failing closed (client.rs
  secure_connection). A fleet relay must therefore be a dumb byte
  pipe that never strips or rewrites rendezvous messages; if we ever
  run our own hbbs, pin its key in the clients. (Not in the first
  build's path: direct mode has no rendezvous messages at all.)

Findings (b) — Pre-warming: warm the tunnel, not the RustDesk session

Fleet side today:

• The broker already pre-warms: serve.rs:592–606 spawns a thread
  that fills the pool for PORT_LAPTOP_AGENT at startup, through the
  same checkout accounting as requests. TunnelPool is per-port,
  DEFAULT_POOL_SIZE = 50 (serve.rs, pool constants ~L66–L96),
  overflow = idle → grow → 300 ms wait → grow anyway.
• But pooled conns are wrapped as HttpConn and spent by
  do_request as HTTP request/response exchanges (serve.rs
  Broker::open_conn / do_request); the broker's wire protocol
  (serve_session, serve.rs:444ff; client.rs hello + JSON frames)
  has no raw-stream mode. A desktop needs a new capability, not a
  new pool.
• Below that layer everything needed already exists:
  tunnel.rs:140–193 connect_tunnel returns a raw TcpStream from
  a CONNECT and reads the proxy response byte-wise precisely so
  payload bytes behind the headers are not eaten — i.e. it was built
  splice-safe. One explicit hazard: tune() applies
  TUNNEL_READ_TIMEOUT = 660 s (tunnel.rs:20–23, 195–199); a desktop
  splice must clear/raise that or long sessions die at 11 minutes —
  forward.sh sessions demonstrably run indefinitely once unmolested.

RustDesk side — which endpoint, and what a bare pre-connect does:

• On a TCP-only path the pool/splice target is the host's
  direct-access port, 21118 — direct-server = Y,
  direct-access-port default 21118 (RENDEZVOUS_PORT + 2;
  rendezvous_mediator.rs get_direct_port/direct_server,
  ~1344–1424; advanced-settings doc above). Do not confuse it with
  hbbs's WebSocket port, which is also numbered 21118 but lives on
  the server host, not the controlled laptop. The relay port 21117 is
  not a host endpoint at all — it is hbbr's, and relay mode would
  require the host to dial out to a fleet-run hbbr and pair by uuid
  (option C below), which the direct port makes unnecessary.
• A bare TCP connect to 21118 is accepted and the host speaks first:
  create_tcp_connection (secure=false → handshake no-op) →
  Connection::start → on_open immediately sends Hash{salt, challenge} and waits. No client protocol bytes are required
  first; the Hash simply sits in the socket buffer until a real
  client is spliced in and reads it, and the client's first frame (an
  empty Message from its non-secure secure_connection fallthrough)
  is a no-op the host discards. So a pre-opened socket is
  protocol-valid to hand over.
• What breaks naive pre-warming (pool-50 for RustDesk sockets):
  1. ~30 s silent-idle death. A connection that says nothing is
     closed when last_recv_time exceeds SEC30 in the connection
     loop (connection.rs:1188–1191; comment at :85–89). The 180 s
     LOGIN_GRACE (connection.rs:90) only covers connections that
     are talking but not yet authorized. A pre-warmed idle socket
     therefore lives < ~30 s and needs constant churn.
  2. Unauthorized-connection caps. MAX_UNAUTHORIZED_CONNS = 64
     globally and MAX_UNAUTHORIZED_CONNS_PER_ADDR = 16
     (connection.rs:91–106, admit_unauthorized at :125ff), counted
     before the handshake, evicting the oldest on overflow. All
     fleet-forwarded connections appear from one address, so a pool
     of 50 is impossible and even ~16 idle pre-connects would crowd
     out real connection attempts.
  3. Each pre-connect is a real host-side Connection: visible in
     the host's connection manager, mints a fresh challenge, posts a
     connection audit record. Churn is user-visible on the laptop.
• Conclusion: the expensive, bursty part of starting a desktop
  today is tunnel establishment (CONNECT through the egress proxy;
  60 s handshake timeout budget in tunnel.rs:19, and seconds-long
  opens under burst per serve.rs's pool comments) — exactly what the
  existing pooled/preconnected tunnels already amortize. The
  RustDesk-layer connect on an already-warm tunnel is one local RTT
  plus the Hash the host sends unprompted. Pre-warm tunnels (have
  it), splice on demand (build it); do not pre-warm RustDesk
  sockets. If measurement later proves the RustDesk connect itself
  matters, a cache of 1–2 direct sockets refreshed every < 20 s is
  the maximum sane size — never 50.

Findings (c) — Concurrency: one desktop, many viewers, zero isolation

A RustDesk host is a single controlled desktop with a multi-connection
server bolted on: Server owns one set of shared services (display,
audio, clipboard, input) and a map of Connections (server.rs
new(), ~125–180); every authorized connection subscribes to the same
services and its input lands in the same session. So yes, multiple
clients can be connected simultaneously — all viewing and controlling
the same desktop, with interleaved mouse/keyboard and no locking,
partitioning, or per-client anything. That is a collaboration feature
and a footgun; fleet must not mistake "second connection" for "second
desktop".

Same shape elsewhere, for contrast:

• KRDP (krdpserver, KDE): a SessionController wraps each RDP
  connection, but every session captures the same running Plasma
  session (Portal or Plasma screencast of that session);
  --virtual-monitor adds a virtual output to that same session.
  Shared desktop, same as RustDesk.
• x11vnc / TigerVNC scraping (x0tigervncserver): attach to the
  live X display :0; every viewer shares it (VNC "shared" is the
  natural mode).
• TigerVNC virtual (Xtigervnc/vncserver) and xrdp+sesman: the
  different model — the server creates a session per login
  (display :N, port 5900+N; xrdp's sesman authenticates and starts
  a per-user X session, reconnect returns to that same session).
  Concurrency there means separate desktops, not shared control.

Findings (d) — A desktop per connection

RustDesk cannot do this. It captures the session that exists —
on Linux, screen capture runs through PipeWire / xdg-desktop-portal
in a logged-in session (and the service can serve the login screen,
which is the part that keeps biting us) — and has no notion of
creating a session per connection. Any per-connection desktop has to
come from a different server or from compositor orchestration under
RustDesk-like tooling:

Sharing data across any of these is the easy half — same filesystem,
explicit browser profile dirs per connection
(chromium --user-data-dir=… per instance avoids profile-lock
contention) — the hard half is session lifecycle: who creates,
resizes, reaps, and logs into each desktop.

Options

• A. Status quo — keep forward.sh (dedicated SSH forward) as the
  desktop path. Works, proven; but permanent second transport, no
  fleet pooling/preconnect, no services-map presence, SSH credential
  path maintained for one port.
• B. Broker raw-splice to the direct port (recommended first
  build) — fleet proxy gains a splice mode: sandbox-side listener
  on 127.0.0.1:21118, each accept checks out a pooled tunnel
  (connect_tunnel) to laptop:21118 and copies bytes both ways.
  Secrets in broker: none (findings (a)). TCP-only compatible by
  construction. Keeps the working RustDesk physical-desktop experience
  and today's direct-IP client configuration, swapping only the pipe
  under it (SSH → fleet).
• C. Fleet-run hbbs+hbbr, force relay — the "real" RustDesk
  topology, and the only one with RustDesk-layer E2E encryption +
  host-key verification. Costs: running/pinning a rendezvous+relay,
  host registration traffic (21116 TCP/UDP) that our sandbox path
  can't carry (servers would have to live laptop-side or on a
  UDP-reachable host), force-relay config on both ends, and the
  fail-open downgrade caveat to police. Revisit if direct-path
  confidentiality ever has to come from RustDesk itself rather than
  the tunnel.
• D. Replace RustDesk with KRDP/xrdp now — premature: throws away
  the working physical-session path to solve the virtual desktop
  problem, which is separable and later (findings (d)).

Recommendation — first build (smallest useful step)

Implement B, in this order:

1. Register the service: add a rustdesk entry for port 21118
   to the fleet services table (fleet-config Config::services,
   currently 4799/4800/4801) so the broker's routing/hello map knows
   the port exists.
2. Add a raw splice capability to fleet-proxy (new broker
   capability beside ProxyServe, negotiated in the hello — proto-0
   brokers simply don't offer it):
   • client verb (name TBD, e.g. fleet proxy desktop) ensures the
     broker, then listens on sandbox 127.0.0.1:21118;
   • per accepted connection: pool_get(21118) → bidirectional copy
     between the local socket and the tunnel stream; no HttpConn
     wrapping, no token file, no request framing; the tunnel is
     spent, never returned to the pool;
   • splice streams must not inherit TUNNEL_READ_TIMEOUT
     (660 s) — clear read/write timeouts on spliced conns
     (tunnel.rs tune() is HTTP-shaped; make it a caller choice).
3. Secrets posture, stated in code comments and docs: the broker
   and the splice hold no RustDesk credential of any kind; the
   permanent password is typed into the RustDesk client on the
   sandbox and its proof travels inside the spliced bytes; laptop
   21118 must remain reachable only through the fleet tunnel /
   loopback-class paths (direct mode has no transport encryption of
   its own — findings (a) caveat), and the laptop permanent password
   must be long and random.
4. Keep forward.sh untouched as fallback until the splice
   proves out; then retire it and the last routine use of the raw SSH
   forward with it.

Acceptance for the first build:

• Desktop session starts through the splice with the laptop's
  existing direct-IP RustDesk config; start latency measured against
  the forward.sh baseline (expect tunnel checkout, not tunnel
  establishment, to dominate).
• A session idles and works well past 30 minutes (no 660 s tunnel
  timeout, no mux-style freeze of the RustDesk: login succeeds but no video — PipeWire ScreenCast negotiation stalls under Hyprland (amd-6) #5509 kind).
• Broker logs/config contain no RustDesk password or proof material
  (they structurally cannot — verify by inspection).
• Two simultaneous splices both land on the same laptop desktop —
  observed and documented as the shared-desktop semantics of
  findings (c), not treated as a bug.

Explicitly not in the first build: RustDesk-socket pre-warming
(findings (b) — at most a later, measured 1–2 socket / < 20 s-refresh
experiment), fleet-run hbbs/hbbr (option C), and per-connection
virtual desktops (findings (d)).

Next steps

• Land the first build above as its own implementation issue
  (services-table entry + splice capability + timeout fix).
• Measure splice vs forward.sh connect latency and long-session
  stability; only then decide if a 1–2-socket RustDesk pre-warm
  experiment is worth its churn.
• Spike (separate design issue): per-connection virtual desktop —
  headless Wayland compositor per connection (kwin_wayland --virtual + KRDP) vs xrdp/TigerVNC-virtual as the X11 stopgap,
  with per-connection browser profile dirs; CRD/amd-crd stays
  the fallback until the spike proves out.
• Decide the end-state for option C (fleet hbbs/hbbr) if a
  future requirement wants RustDesk-layer encryption instead of
  tunnel-layer confidentiality.