Daemon lets a second deploy preempt an in-flight deploy on the same port (no per-port lease); contention misreported as 'serial driver may be wedged' #1461

Correction: the first version of this issue blamed two concurrent daemons. That was wrong. The daemon log shows single-flight working: every extra spawn logged another fbuild-daemon already owns 0.0.0.0:63518; yielding (single-flight) and exited. The real defect is that the one daemon lets a second deploy take over a port while a first deploy is still flashing it.

What happened

Three FastLED checkouts (fastled2, fastled-wt-4583, fastled-wt-4583-base) all ran bash autoresearch esp32c6 ... against one ESP32-C6 on /dev/ttyACM1, through the same daemon on port 63518. From ~/.fbuild/prod/daemon/daemon.log (UTC):

01:29:41  deploy: selected /dev/ttyACM1                            <- fastled2
01:29:41  port closed client_id="deploy_preemption"
01:29:43  flashing .../fastled2/.../firmware.bin via esptool       <- flash A running
01:30:04  deploy: selected /dev/ttyACM1                            <- fastled-wt-4583-base, 23 s into flash A
01:30:04  port closed client_id="deploy_preemption"                <- accepted, not refused
01:30:04  native verify-flash failed ... Unable to acquire exclusive lock on serial port
01:30:05  esptool verify-flash failed: Could not open /dev/ttyACM1, the port is busy
01:30:05  flashing .../fastled-wt-4583-base/.../firmware.bin      <- second esptool on the same port
01:30:08  USB re-enumeration: port /dev/ttyACM1 not available after 3s
01:30:24  deploy: selected /dev/ttyACM1                            <- fastled2 retry, same pattern again

What the clients saw: flash A's esptool died with esptool failed (exit code -9) / Deploy failed (fbuild), and a serial attach got attach failed: open_port(/dev/ttyACM1) exceeded 3s; serial driver may be wedged. The driver was never wedged; the port was mid-flash for another request.

Root cause (fbuild 0433e964)

• crates/fbuild-daemon/src/handlers/operations/deploy.rs (~L437) calls serial_manager.preempt_for_deploy(port, "deploy", request_id) unconditionally and discards the result (let _ =). It never asks whether another deploy already holds the port.
• crates/fbuild-serial/src/preemption.rs PreemptionTracker::preempt does ports.insert(...), so a second deploy silently overwrites the first deploy's preemption record instead of being refused.
• crates/fbuild-daemon/src/device_manager.rs already has an exclusive device lease (exclusive_lease, preempt_device, DeviceLeaseError), but the deploy path doesn't use it, so deploy-vs-deploy and deploy-vs-monitor on one port are not serialized.
• handlers/websockets.rs await_ws_serial_open_port then reports the resulting contention as "serial driver may be wedged".

Expected

The daemon is the single owner of every device, so a request for a port that another request holds should come back right away with a clear "in use" error, not start a second flasher:

port /dev/ttyACM1 is in use: deploy by request <id> (project /home/.../fastled-wt-4583-base) since 01:29:41

Ask

1. Take an exclusive per-port lease for the whole deploy (preempt → verify → write → re-enumerate → clear). A second deploy or reset on a leased port either fails fast with the holder's identity, or waits in FIFO order with a bounded timeout if the caller opts in. It must never preempt an in-flight deploy.
2. Make PreemptionTracker::preempt refuse (return an error) when the port is already preempted by a different request, and stop ignoring preempt_for_deploy's result.
3. A serial attach on a port with an active deploy lease should return "port in use by deploy ", not "serial driver may be wedged". Keep "wedged" for a real open timeout with no holder.
4. A regression test: two concurrent deploy requests on the same port; the second is rejected (or queued) and the first's flash completes.

Related

• daemon retains an exclusive serial handle after deploy; next open gets EBUSY and is reported as "serial driver may be wedged" #1429: the same "wedged" misreport when the daemon itself keeps a stale handle after a deploy.
• deploy/port: accept a USB serial where --port takes a node path (unattended benches renumber) #1428: identifying a board by USB serial; a lease keyed by USB serial would also survive re-enumeration.