Skip to content

RustDesk: login succeeds but no video — PipeWire ScreenCast negotiation stalls under Hyprland (amd-6) #5509

Open

Description

@developing-today

Symptom

RustDesk login to the laptop (amd-6) now succeeds, but the session never delivers video: the client sits at "Connecting…" and then fails with "Connection error: Timeout". Zero PipeWire frames ever flow.

What is proven working (not the problem)

  • Credentials: the password reset is complete — Bitwarden rustdesk.amd-6-password is now accepted by the host (config field 118 chars, prefix 01; RustDesk service active on :21118). This is not a credential issue.
  • Route / transport: the connection reaches the host and authenticates; this is not a routing issue.
  • Client rendering: the sandbox client's GL/render path was already fixed separately (llvmpipe, direct rendering); this is not a client-render issue.

Evidence (2026-11-04 reset run)

  • Post-auth, the session reaches the host's screen-share picker.
  • The picker was automated once end-to-end (ydotool + hyprctl geometry on hyprland-share-picker; "Remember this choice" ticked) and the share was granted.
  • Despite the granted share, zero PipeWire frames flow: client shows "Connecting…" → "Connection error: Timeout".
  • Host log shows the connection closed on Timeout after encoder selection (AV1) with no frames delivered.

Evidence files:

  • ~/workspace/rd-work/rd-portal3.jpg — live screenshot of the host screen-share picker
  • client-timeout-error.png (sandbox) — client-side timeout error

Diagnosis

The stall is in ScreenCast Start / PipeWire negotiation on the laptop's Hyprland/portal stack: authentication and share consent complete, encoder selection (AV1) completes, but the PipeWire stream never starts producing frames. RustDesk's own UI flags Wayland capture as experimental.

Family / cross-link

Same family as #5415 (KRdp dead under Hyprland — its KWallet user store never loads): the laptop's remote-desktop capture paths are rotting under the Hyprland session in different places. #5415 explicitly scoped RustDesk out ("RustDesk is the other fallback … out of scope here"); this issue is that separate failure.

Acceptance

  • A RustDesk session from the sandbox client displays the laptop desktop (frames actually flowing, not just auth + consent), or
  • a documented switch of the laptop's remote-desktop path to a working capture stack, with the RustDesk/Hyprland-PipeWire stall recorded as the reason.

Activity

  1. developing-today commented on Nov 4, 2026

    @developing-today
    ContributorAuthor

    RESOLVED 2026-11-04 — video leg fixed; acceptance met with screenshot proof.

    Root cause — two stacked defects, neither in RustDesk's capture code

    1. xdg-desktop-portal-hyprland was wedged on amd-6. Its main thread spun at 100% CPU (utime+stime delta 299 ticks / 3s, thread state R) and the daemon was D-Bus-deaf: ScreenCast Start never received a Response (wl-stage:start:no-response: in the RustDesk server log), session Close timed out in the frontend ("Failed to close session implementation: Timeout was reached"), and even busctl get-property org.freedesktop.portal.Desktop /org/freedesktop/portal/desktop org.freedesktop.portal.ScreenCast version timed out. CreateSession/SelectSources intermittently squeaked through, which made it look like a PipeWire negotiation problem — but no PipeWire node is ever created when Start is never answered. It had arrived wedged (the day's first CreateSession already returned NoReply at 07:37) after the frontend portal was restarted at 06:57 against a backend started at 06:00.
    2. The client route rode the shared SSH ControlMaster. The sandbox→laptop forward was owned by the single mux process that also carries every other fleet session (25-minute cargo-test output, 1700s sleep-pollers, build watches). Under mux contention the forwarded stream froze in both directions: server-side Connection closed: Timeout at ~100–170s per connection, client-side "Connection error: Timeout", endless auto-reconnect — and each reconnect minted a new portal session and a new share picker on the desktop. (A --server process that had absorbed this churn later failed CreateSession with NoReply even against the freshly restarted daemon, until the service stack was restarted clean.)

    Fix

    • systemctl --user restart xdg-desktop-portal-hyprland (later also the frontend portal + rustdesk service, to clear a morning of poisoned session state). Post-restart: Start completes <1s, GStreamer pipeline confirms PLAYING, wpctl shows a live rustdesk node ← capture_1 [active]. Hyprland itself was never restarted.
    • Dedicated SSH forward for RustDesk (sandbox tooling, ~/tools/rustdesk-gl/): new forward.sh (ControlMaster=no / ControlPath=none); launch.sh now cancels any mux-owned :21118 forwarding and always runs the dedicated forward; auth.sh corrected to the proven no-click paste (the dialog auto-focuses its field; the old baked click coordinate defocused it); picker grant helper preserved as share-grant.py; README "Video leg" rewritten with this diagnosis.
    • No flake change needed and none made: no NixOS-side defect was found — xdph 1.4.1-git + Hyprland 0.56 stream correctly once the daemon is not wedged, and the transport defect was in sandbox-side tooling. Deliberately not pinning xdph to the older 1.3.6 in the store: the wedge did not recur across ~10 Start attempts post-restart, and there is no evidence the older build prevents it.

    Acceptance evidence

    Fresh sandbox-client session (launch.sh + auth.sh + picker grant): streamed the real laptop desktop (Zed workspace + host connection-manager panel) continuously — per-frame pixel diffs nonzero for 5.5 minutes, an injected client-side mouse move produced a 26% frame change (input round-trip works), and the host's own cm panel read "Connected 00:06:47" and counting in the final frame, versus the ~170s death ceiling all morning. Screenshots: ~/workspace/rd-work/proof-desktop-frames.png (+ -early.png, series fp-1..fp-9).

    Noise explicitly not chased: RustDesk 1.5.0 --server processes panic on exit ("cannot access a Thread Local Storage value during or after destruction", 22× that day) — fires during teardown after the connection's fate is decided; the service respawns. Client 1.4.5 ↔ server 1.5.0 streams fine.

    Note: a final re-validation of the shipped scripts was cut short when amd-6 rebooted at ~09:39 CST (sddm greeter up, no graphical login since). auth.sh as-shipped had already submitted successfully (rc=0) seconds before the reboot; the fix itself was proven end-to-end before it, as above. Nothing in the fix depends on the reboot.

  2. developing-today commented on Nov 4, 2026

    @developing-today
    ContributorAuthor

    REOPENED — regression: the reconciled RustDesk password reverted after the 09:39 reboot; repair re-applied and re-proven 13:21 CST, durability fix still owed

    This issue was closed this morning on the strength of two acceptances: (1) the Bitwarden rustdesk.amd-6-password value is accepted by the host (permanent-password storage reconciled), and (2) video streams. Acceptance (1) silently regressed within hours; (2) is unaffected.

    Regression evidence (2026-11-04, amd-6)

    • 13:12–13:13 CST: two client auth attempts with the vault value were rejected by the host.
    • Live ~/.config/rustdesk/RustDesk.toml at inspection held a 214-char password field with the legacy 00 prefix (the double-wrapped form) — not the reconciled 118-char 01 storage written this morning. File mtime 09:39:27, size back to 1256.
    • The reverted file is not a byte-restore of any backup: md5 differs from RustDesk.toml.bak-prewrite at identical length and identical key set; salt unchanged; enc_id refreshed — i.e. the field was re-serialized, not restored.

    Mechanism (from host evidence, not theory)

    The toml is not declarative state: it is a plain regular file (no symlink), it does not appear in home-manager's generation file set, dezren39/darwin-nix contains no rustdesk configuration at all, and the rustdesk.service user unit is a hand-written file dating from Oct 23. The generation-switch activation (09:39:15) did not write it, and the later 10:36 home-manager activation provably did not touch it either.

    The write at 09:39:27.117 matches to the second the post-reboot recovery launches of rustdesk --server under sudo with DISPLAY=:0 (journal: PIDs 9762/9918, both dying with cannot open display: :0). A rustdesk process starting in that foreign session context re-serialized the password field into the legacy double-wrapped form. So: the switch reboot created the occasion; the writer was the stray --server start, not NixOS. Corollary, verified today: a plain systemctl --user restart rustdesk.service does not revert the field.

    Repair re-applied (13:17–13:21 CST) and proven

    Same procedure as the original fix: setter-exact storage computed on amd-6 (libsodium via ctypes; vault value by name, piped on stdin only, channel integrity asserted by sha256, never printed/logged), self-verified by decrypting the computed field back to SHA256(pw ‖ salt), single-line splice with the service stopped (backup RustDesk.toml.bak-step1-20261104), service restarted.

    Proof, config-shape alone explicitly not accepted:

    • Sandbox client (~/tools/rustdesk-gl/launch.sh + auth.sh): password accepted, session advanced to the host's Wayland share step; share granted; desktop streamed (Zed session visible in the client).
    • Post-restart, the live field decrypt-verifies against the vault value's hash and is byte-stable.

    Durability fix spec (why this stays open)

    1. Self-healing assertion: hold the reconciled storage in the fleet SOPS layer for amd-6 and add a guard (user timer or ExecStartPre wrapper on rustdesk.service) that decrypt-verifies the live password field against the vault value's SHA256(pw ‖ salt) and, on mismatch, rewrites just that line from the SOPS-held storage with the service stopped. Verification is a boolean/decrypt check only — values never printed or logged.
    2. Operational rule until (1) lands: never launch rustdesk --server via sudo or with a foreign DISPLAY/session env; recovery goes through systemctl --user restart rustdesk.service only.
    3. Re-acceptance: after the next generation switch + reboot, the vault value must authenticate on the first attempt with no manual repair.
  3. added
    work/assignedQueue worker lifecycle state
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    on Nov 5, 2026
  4. dezren39 commented on Nov 5, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  5. added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 5, 2026
  6. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/queuedHas a queued task in the Supabase agent queue
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 5, 2026
  7. 4 remaining items

  8. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    on Nov 5, 2026
  9. dezren39 commented on Nov 5, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  10. dezren39 commented on Nov 5, 2026

    @dezren39
    Member

    🤖 Agent note · state: handed off

  11. dezren39 commented on Nov 5, 2026

    @dezren39
    Member

    🤖 Agent note · handed off

  12. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 5, 2026
  13. dezren39 commented on Nov 5, 2026

    @dezren39
    Member

    🤖 Agent note · state: picked up

  14. dezren39 commented on Nov 5, 2026

    @dezren39
    Member

    🤖 Agent note · triage complete (task 10077)

    Verdict: #5509 is still OPEN, but its current blocker is no longer PipeWire video capture. The 2026-11-04 issue comment records working desktop video after restarting the wedged portal and using a dedicated RustDesk SSH forward; the issue was reopened for password persistence after reboot, with video explicitly unaffected.

    Code path: the vault reassertion is implemented on main: PR #6140 added a boot-time assertion, PR #6241 added bws to the unit PATH after gen 537 failed, and current main commit da89bd8d pins sh/cat/bws PATH inside the runuser child after gens 537/538 failed. No open implementation PR remains.

    Remaining acceptance / ordering: keep the issue open until the final main version is activated on amd-6 and, after a generation switch/reboot, the vault password authenticates on the first attempt without manual repair. The current evidence proves the code path and a simulated regeneration, not post-final-fix host activation. This is deployment validation, not a remaining code fix. I did not switch or reboot the host in this triage run. Once the acceptance is recorded, close #5509; if it fails, route the exact new failure against the deployed generation.

  15. added
    status/triagedTriaged and routed (see agent note)
    and removed
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

status/triagedTriaged and routed (see agent note)task/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions