Turn dies silently mid-stream: session stays busy forever, queued prompts never run, no abort/status endpoint #59559

Summary

A model turn can die silently mid-stream. Nothing is logged, the process goes idle, and time.idle never appears — but the session stays busy forever. Every prompt sent afterwards is accepted (HTTP 200) and queued behind the dead turn, yet never processed. There is no way for a client to detect or abort it: on this build POST /api/session/<id>/abort and GET /api/session/<id>/status both return 404.

Recovery requires killing the sidecar and starting fresh sessions.

Environment

• Desktop app 2.1.4, using its SSH sidecar path: opencode serve --service on loopback 49374
• Servers: Ubuntu 22.04 and 24.04 (two hosts), plus a macOS client
• Providers: several; the failure is not tied to one provider/model as far as I can tell
• No proxies, no VPN in the loop for the affected hops

Reproduction pattern

Not deterministic per request, but reliably reproducible under a burst:

• Dispatch 10 independent conversations with concurrency 3 on one host → 4 of 10 sessions wedged
• The same kind of work at concurrency ≤2 on a second host stayed clean in the same period
• Resumption after a sidecar restart (several sessions resuming at once) reproduced it too
• A host left idle for hours reproduces nothing

So it looks load/timing related rather than input related.

What the logs show

• Sidecar log: no error, no warning at the moment the turn dies; the last entries are normal stream activity
• Process: sidecar is alive, no model child process activity, CPU idle
• Client: the session shows busy; time.idle is never emitted for that turn
• Sending a new prompt returns HTTP 200 (stored) and produces zero processing

Impact

For interactive use it is recoverable (restart), but for automation it is the worst failure mode: the queue silently accepts work that will never run, so a dispatcher that trusts the HTTP 200 looks healthy while doing nothing.

Requests

1. Emit a terminal marker when a turn's stream dies. If the upstream stream ends without a finish/error signal, set the session back to idle (or a failed/interrupted state) with an error field, instead of leaving it busy forever. A watchdog on the turn (no events for N seconds) would catch this.
2. Add abort/status endpoints. POST /api/session/<id>/abort and a read-only GET /api/session/<id>/status (e.g. idle|busy|failed, current turn started-at, last event-at) would make detection and recovery possible without killing the process.
3. Make queued prompts recoverable. Either process them once the dead turn is cleared, or reject them explicitly while the session is stuck — anything is better than 200 + silence.
4. Optional: a client-visible heartbeat per running turn would let dispatchers detect wedging without polling the DB.

Workaround, for anyone hitting this

• Keep concurrency ≤2 per host and stagger overlapping work
• Detect + recover: kill the sidecar, respawn it from the desktop app, then continue in fresh sessions seeded with the previous progress (this recovery path worked 6/6 times here)
• Do not trust a 2xx from a prompt post as proof of processing

Happy to provide sidecar logs and a timestamped capture of a wedged session if that helps.

Related: #59556 (hang with AGENTS.md present) looks like a different trigger for a similar symptom — this report is specifically about the missing terminal state and the unusable queue afterwards.