[Bug]: Grok delegated child stays "running" after its turn ends; parent never notified #15888

Activity

1. juliusmarminge commented on Oct 5, 2026

   

   juliusmarminge

   on Oct 5, 2026

   Member

   More actions

   Note

   Grok responding on behalf of Julius.

   Triage

   Thanks for the thorough report and the DB detail, @topitopongsala!

   What I found

   • The session/prompt failed / errorTag: Interrupt line looks like noise, not the failure. makeXAiPromptCompletionRuntime settles the turn from turn_completed / x.ai/session/prompt_complete, then interrupts the still-open RPC, and the native log records that interrupt. That's why every turn, including the ones that complete normally, ends the same way.
   • On the stuck children, what's missing is a terminal provider-turn event. The final assistant item is written from session updates on its own, so the transcript can look finished while the run and provider turn stay running. The parent wake is tied to the child run becoming terminal, so it never fires.
   • The likely path is Grok's deferred finalize (deferFinalizeForBackgroundWork). On prompt settlement, if a non-persistent background tool or a subagent is still pending/running, the adapter marks promptSettled and returns without emitting a terminal or arming the 3s debounce. A later completion is supposed to call rearmDeferredFinalize, but two cases never do:
     • task_completed is ignored until promptSettled (finishRegisteredBackgroundTool). If it arrives first, it's dropped, the tool stays running, and settlement then pins the run.
     • A subagent still running/pending pins the run the same way, with no timeout (unlike hydration at 60s and injected reports at 25s).
   • task_cancel still reaches a live turn here (Grok hard-restarts and interruptTurn finalizes it as interrupted), which matches what you saw and is why the finished result gets discarded. That's different from fix(server): Stop ends a run whose turn its adapter already settled #15298, where the adapter had already dropped its active turn. It also doesn't look like Feature: Parent is never notified when a delegated child finishes a follow-up turn (V2) #13490, fix(server): recover runless native child provider turns #14434, or [Bug]: Remote (SSH-launched) server dies on client disconnect, delegated subagents reported as cancelled #15608.

   If you catch a child while it's still stuck (before task_cancel), this would show which case it is:

   SELECT type, status, count(*)
   FROM orchestration_v2_projection_turn_items
   WHERE thread_id = '<stuck child thread id>'
   GROUP BY 1, 2;

   A tool or subagent row still running/pending after the final assistant message points to the deferred-finalize pin. If every item is terminal and the provider turn is still running, the settlement callback itself was dropped, which would be a different gap.

   Likely fix area

   • One option is recording an early task_completed instead of dropping it, so it's applied once the prompt settles.
   • Another option is a timeout on the deferred finalize for subagents, similar to the existing hydration and injected-report timeouts.
   • Both live around deferFinalizeForBackgroundWork, finishRegisteredBackgroundTool, and rearmDeferredFinalize in the Grok adapter.

   A maintainer will decide on the fix direction.
2. juliusmarminge

   added

   bugSomething is broken or behaving incorrectly.Something is broken or behaving incorrectly.

   via-triageFiled through npx t3 triageFiled through npx t3 triage

   on Oct 5, 2026
3. filipproch commented on Oct 5, 2026

   

   filipproch

   on Oct 5, 2026

   More actions

   Same bug here on Linux. Two of my Grok children were still stuck when I read the triage, so here is the query you asked for, plus a census of which end signals T3 loses.

   Setup: T3 Code desktop 0.0.45-main.8bc40b4e0, Grok CLI 1.0.46 (stable), children on grok-4.7-build-fast with reasoning xhigh and full access, started by delegate_task (async) from a Claude parent.

   The triage query on two stuck children

   Both children had sent their final answer about 20 minutes earlier. For both, Grok's own session log has turn_completed with stop_reason: "end_turn" and turn_ended with outcome: "completed".

   child A                              child B
   assistant_message|completed|6        assistant_message|completed|6
   command_execution|completed|16       command_execution|completed|23
   command_execution|running|7          command_execution|failed|1
   dynamic_tool|completed|14            command_execution|running|2
   file_change|completed|1              dynamic_tool|completed|25
   file_search|completed|11             file_change|completed|1
   reasoning|completed|29               file_search|completed|8
   user_message|completed|1             reasoning|completed|31
                                        user_message|completed|1
   

   There are no subagent rows. Every open row is a command_execution for a Grok background shell (rawOutput.type: "BackgroundTaskStarted"). So this is the deferred-finalize hold, in the background-tool branch of hasDeferredBackgroundWork. In child A, all seven commands had exited 0, and Grok sent task_completed for each one before the turn ended. T3 even wrote the final output text into those rows, but their status stayed running.

   Which end signals T3 honors

   I matched every Grok background command in my runs since 2026-09-30 (81 commands) against how Grok reported its end in its session updates.jsonl, and checked whether T3 closed the row.

   How Grok reported the end	Commands	Rows T3 closed
   Single-task TaskOutput poll (Result) on a UUID task id	24	23
   Kill (rawOutput.type: "KillTask", title kill <id> (killed))	24	0
   Multi-task poll (TaskOutput.MultiResult)	14	0
   Single-task poll on a call-* task id	8	0
   task_completed only, with no poll or kill	8	0
   No end signal seen	3	0

   So the only path that works today is a single-task poll on a UUID id. I see four gaps at 8bc40b4e0. The two gates are unchanged on current main.

   1. finishRegisteredBackgroundTool returns early while !context.promptSettled. In the stuck runs, task_completed arrived before turn_completed, so this is the first branch of your analysis.
   2. extractXAiKilledBackgroundTasks requires a title that contains kill_command_or_subagent or variant === "kill". Grok 1.0.46 sends KillTask and titles the finished call kill <id> (killed).
   3. TaskOutput hydration accepts only UUID task ids. Commands that Grok moves to the background itself, after the blocking timeout, get call-<uuid>-<n> ids.
   4. Nothing on the v2 path reads TaskOutput.MultiResult. The v1 helper XAiBackgroundTasks.ts handled it, but now only its own test imports it.

   Rates

   These are runs since 2026-09-30 that stayed open more than 10 minutes after their final assistant message.

   Provider	Runs	Stuck
   grok	66	19
   claudeAgent	609	0
   codex	134	1

   Of the Grok runs, 35 started no background command, and none of them got stuck. Of the 21 runs that started at least one, 16 stayed open until someone cancelled them.

   On the fix direction

   Fixing the four parsers would cover the shapes Grok sends today. The run would still hang the next time a shape changes, because nothing bounds the hold. The v1 Grok adapter had a liveness watchdog (10 minutes, or 30 with an active tool), and v2 has none. So Stop and a server restart are the only ways out.

   I'd rather background commands never held the run. turn_completed would end the run, and commands still running would stay as pending work. backgroundWorkHoldsCompletion in orchestrationV2PendingBackgroundWork.ts already treats commands that way for completion alerts, because a dev server can run for hours. Since #14488, Grok's reply to a finished background command opens its own run, so I don't think the hold buys much for commands. Subagents and monitors could keep the hold, with a timeout like the 60 s and 25 s ones that already exist.
4. lukemaj

   added a commit that references this issue on Oct 10, 2026

   fix(server): carry upstream Grok fixes for stuck background commands …

   ...

   8aec64b
5. al0x99 commented on Oct 10, 2026

   

   al0x99

   on Oct 10, 2026

   More actions

   Same bug on macOS with Grok CLI 1.0.50 and T3 Code 0.0.46-nightly.20261009.2861. Two delegated Grok children stayed running for about 16 hours after their final answer, until task_cancel.

   Every row left open was a Grok background shell. Grok had reported the end of each one, in shapes main does not apply:

   • two shells sent task_completed about 17 minutes before the root turn_completed, so the !promptSettled gate dropped them;
   • one shell was stopped with kill_command_or_subagent, which returned rawOutput.type: "KillTask" with the title kill <id> (killed).

   After the root turn, Grok ran two task-completed-* wake turns. Their frames stayed buffered behind the held run.

   Wire method on 1.0.50. Grok's own updates.jsonl stores task_backgrounded and task_completed as sessionUpdate values on _x.ai/session/update. On the wire they are separate notifications. I drove grok agent --always-approve stdio (1.0.50) from a minimal ACP client with sleep 8 in the background, and recorded _x.ai/task_backgrounded and _x.ai/task_completed as their own methods, with turn_completed on _x.ai/session_notification. The _x.ai/session/update shape looks like how Grok persists its log, not what it sends to T3.

   Replay of our trace. I turned the trace into a replay fixture with the wire methods above and neutral commands and outputs. It is not in a PR; I can send it if useful.

   Tree	Result
   main (c77a7b7ee)	Times out after 60 s. Run 1 and three command rows stay running.
   main + #17144	Completes in about 1 s. Only Grok's wake reply text is missing.
   main + #17504	Completes, but the wake reply text is missing too. Separately, the #17504 head fails grok_background_bash, grok_background_bash_fast_wake and grok_monitor with no_finish_armed.

   The missing wake reply is a separate problem, the one #17144 lists under "Left out". Once any task was handled in the turn, the adapter drops every later agent chunk as chatter, including Grok's reply in a wake turn for a different task. I opened #17793 for that alone. It keys the chatter rule on the task named in the task-completed-<id> prompt id, and includes a fixture recorded live from Grok 1.0.50.

   Neither #17144 nor #17504 puts a time limit on the hold.
6. letrandat commented on Oct 13, 2026

   

   letrandat

   on Oct 13, 2026

   More actions

   Still happening on T3 Code 0.0.46-nightly.20261012.3021 with Grok CLI 1.0.52 (macOS, Darwin 25.6.0). A delegated Grok child (grok-4.7, full access, async delegate_task from a Claude parent) stayed running for 9h32m after its final answer. The parent was never woken. It only got out because T3 restarted.

   One detail may help narrow the fix. Three background shells ran in the same turn, and each one's task_completed arrived before the root turn_completed. Two closed normally and one stuck. The stuck one is the shell that Grok backgrounded on its own after the 30 s foreground limit, where task_id equals the tool call id. The two that closed were started as background tasks and have UUID task ids.

   From Grok's own updates.jsonl (UTC):

   16:12:17 task_backgrounded tool_call_id=…-143  task_id=01a130e2-e216-…   (row closed)
   16:12:49 task_completed    task_id=01a130e2-e216-…  exit=0
   16:12:53 task_backgrounded tool_call_id=…-145  task_id=01a130e3-6e0b-…   (row closed)
   16:13:14 task_completed    task_id=01a130e3-6e0b-…  exit=0
   16:17:47 run_terminal_command call-…-23 starts (foreground)
   16:18:17 task_backgrounded tool_call_id=call-…-23  task_id=call-…-23      (auto-backgrounded, tool_completed 30008 ms)
   16:18:19 get_command_or_subagent_output task_ids=[call-…-23]
   16:18:39 task_completed    task_id=call-…-23  exit=0
   16:19:55.602 turn_completed prompt_id=t3-xai-prompt-1 stop_reason=end_turn   (events.jsonl: turn_ended outcome=completed)
   

   On the T3 side:

   • session/prompt went from started to failed (errorTag: Interrupt) at 16:19:55.625, the same instant as the final assistant_message.
   • The command_execution row for call-…-23 stayed running. A separate dynamic_tool row titled [bg] … (call-f4f) did complete. The run and the thread stayed running.
   • At 01:52:12 the next day T3 restarted. It sent session/close, marked run 1 cancelled (and the stuck command row with it), then started run 2 with "T3 Code restarted and resumed this turn". Grok replied in 15 s, run 2 completed, and the parent woke.

   So on this build a restart works around it (run 1 still ends as cancelled even though it finished). #17144, #17504 and #17793 are still open, and none of them puts a time limit on the hold. A bounded fallback would cap the damage from the next variant: once the root turn_completed / end_turn has arrived, finalize after N minutes even if background rows are still open. This is the most common way our Grok lanes fail in multi-model runs.
7. lukemaj

   added a commit that references this issue on Oct 14, 2026

   fix(server): carry upstream Grok fixes for stuck background commands …

   ...

   7e71fba
8. paralyses2193 commented on Oct 14, 2026

   

   paralyses2193

   on Oct 14, 2026

   More actions

   A deterministic repro on today's nightly. It takes about a minute and needs no long-running work.

   Setup. T3 Code 0.0.46-nightly.20261014.3118 (Linux desktop), Grok CLI 1.0.52 stable, a Claude parent thread, delegate_task async to grok / grok-4.7 (reasoning low, full access).

   Repro (stuck every time). Delegate this task:

   1. Run sleep 10; date -u +%FT%TZ > ~/bg-done.txt in the background.
   2. Then run sleep 40; echo foreground-done in the foreground and wait for it.
   3. Don't poll, read or kill the background command.
   4. Reply "Both commands ran. Done." and end your turn.

   The background command ends while the prompt is still open, and nothing ever reads it.

   UTC	Source	Event
   23:21:31	Grok	task_backgrounded task 01a13cb8-934e-…, the T3 row [bg] sleep 10; … goes running
   23:21:41	file	bg-done.txt written (command exited 0)
   23:21:41	Grok	task_completed for 01a13cb8-934e-…, exit_code: 0
   23:22:16	Grok	final answer, turn_completed end_turn
   23:22:16	Grok	task-completed-* wake turn, finished at 23:22:29
   23:30:50	T3	still status: running, workState: working, row still running, parent never woken. Ended with task_cancel.

   Control (completes). Same task, but with only the background sleep 45 and an immediate reply. That task_completed arrives after the prompt settles, and the run completes 3 s later. The only difference is whether the end arrives before or after settlement.

   Unit test that hangs on main (5a4e998053). In AcpAdapterV2.test.ts, a flavor with deferFinalizeForBackgroundWork: true and extractBackgroundTaskId registers tool-call-bg-1 as task-bg-1. The test gates prompt, applies applyBackgroundTaskMutation with running and then completed while the prompt is open, and then resolves the prompt with end_turn. It never sees turn.terminal and times out after 120 s. Happy to put this up as a test-only PR if that helps.

   The one-line fix is not enough. If I drop !context.promptSettled from finishRegisteredBackgroundTool, that test passes. But six existing tests fail, the ones that expect one post-finalize continuation for a mid-turn completion that was never handled in-turn (for example "mid-turn unhandled background completion still offers exactly one continuation after finalize"). So the focused fix probably needs to close the row (or ignore it in hasDeferredBackgroundWork) while keeping the midTurnUnreportedCompletedTaskIds mark that drives the continuation offer. That trade-off seemed better left to you.

   Field data from the same machine. On 2026-10-13, two delegated Grok reviews stayed running after their final answer. Every row left open was a [bg] command:

   • One run had two commands that exited at 12:09:31 and 12:12:44. Grok read both with multi-wait (wait_all) (variant: "TaskOutput", results under MultiResult.results[], both status: completed). The run stayed running from the final answer at 12:23:25 until task_cancel at 13:39:51.
   • One run had a command that Grok stopped with variant: "KillTask", titled kill call-bd2 (killed), task id call-bd283fbf-…-24 (the tool-call id, not a UUID). It stayed running from the final answer at 15:17:04 until task_cancel at 16:47:48.

   Both match the shapes listed above, on Grok CLI 1.0.52.
9. 3xcalibur-tech commented on Oct 15, 2026

   

   3xcalibur-tech

   on Oct 15, 2026

   Contributor

   More actions

   Fresh reproduction on Grok Build 1.0.60, preserving the background-command case from #17504 as requested in the closure comment.

   Environment and latest-version check

   Checked on 2026-10-15: macOS 27.0.1 arm64, Node 24.21.0, Grok 1.0.60 (01a8e1c591c0) [alpha], model grok-4.7-build-fast. The CLI's own update check returned:

   {"currentVersion":"1.0.60","latestVersion":"1.0.60","updateAvailable":false,"installer":"internal","channel":"alpha","autoUpdate":null,"error":null}

   The live reproduction used unmodified main 42da166, driving makeGrokAdapterV2 and its real ACP runtime with grok agent --always-approve stdio in a fresh temporary directory. Main advanced to 98bc998e73ad823cc15df6f110fedec10261a5bc during verification; the ACP/Grok adapters and orchestration files are unchanged between those commits.

   Reproduce

   In a full-access Grok thread, send:

   Use only your terminal tool. First run sleep 5; echo EARLY_BG_DONE in the background. Then run sleep 15; echo FOREGROUND_DONE in the foreground and wait for it. Do not poll, read, or kill the background task. Do not use subagents. Reply exactly BOTH_COMMANDS_RAN and end your turn.

   The foreground command keeps the root prompt open while the background command finishes. In my live run the model followed those steps and did not poll the background task.

   Observed on unpatched main

   These are timestamps from the ACP wire capture and adapter events (UTC, 2026-10-15):

   16:22:36.761 _x.ai/task_backgrounded
                   task_id=01a1405f-67d4-7961-a216-b26c88563120
   16:22:41.953 _x.ai/task_completed
                   same task_id; kind=bash; exit_code=0
                   output="EARLY_BG_DONE\n"; will_wake=true
   16:22:55.844 _x.ai/session_notification: turn_completed
                   prompt_id=t3-xai-prompt-1; stop_reason=end_turn
   16:23:02.253 _x.ai/session_notification: turn_completed
                   prompt_id=task-completed-01a1405f-67d4-7961-a216-b26c88563120
                   stop_reason=end_turn
   16:23:05.851 T3 observation: original command still running,
                   output already contains EARLY_BG_DONE,
                   deferred-finalize hold entered, no turn.terminal,
                   no continuation offer, hasPendingBackgroundWork=true
   

   The live observation window was 10 seconds after the root prompt settled; it is not a claim that I waited hours. Both Grok turns had ended by the final observation. A deterministic regression test of this ordering also times out on main for both successful and failed commands.

   Focused fix and before/after evidence

   Retain structured command completion until root prompt settlement, then close the registered tool before checking deferred background work. Keep the authoritative completed/failed status despite BackgroundTaskStarted normalization. Preserve the existing unreported-result continuation; receiving a lifecycle event does not mean the agent consumed its result. Monitor report handling stays on its existing path.

   The same live prompt with the focused patch produced:

   16:29:07.322 Grok task_completed, exit_code=0, will_wake=true
   16:29:22.310 Grok root turn_completed, end_turn
   16:29:22.333 T3 turn.terminal, completed
                   background command completed, output EARLY_BG_DONE
                   foreground command completed, output FOREGROUND_DONE
                   deferred-finalize hold not entered
                   exactly one continuation offered for the unreported result
   

   This live harness stopped after observing root completion and the continuation offer; it did not execute that continuation. The four existing Grok orchestrator replays (grok_background_bash, grok_background_bash_fast_wake, grok_monitor, grok_background_subagent) pass with the patch, as do all 120 ACP adapter tests.

   For the historical 1.0.50 investigation, the original live ACP evidence and the original orchestrator run remain on #17504. Those were runs of the old patched branch, not fresh proof against current main. The unpatched-main evidence above establishes the current failure independently. This narrows the replacement to the early command-completion branch of this issue; it does not claim to fix every subagent, TaskOutput, or KillTask case discussed here.

   Investigated and implemented with GPT-6 Astra (high reasoning effort), T3 Code / Codex harness.
10. wookiist commented on Oct 19, 2026

    

    wookiist

    on Oct 19, 2026

    More actions

    Still reproduces on Grok CLI 1.0.62 with T3 Code 0.0.46-nightly.20261019.3348 (macOS 27.0, arm64). Two of four delegate_task (async, full-access) children to grok-4.7 stayed running after their final answer until task_cancel; the two that only read files completed normally. A few details that I don't think are in the thread yet:

    1. Both stuck rows were auto-backgrounded shells with call-… task ids, matching @letrandat's observation. One child waited on the shell with get_command_or_subagent_output and still got stuck, so the resultTaskIdFromGetOutputTool path (gated on XAI_UUID_RE in xaiAcpExtension.ts) does not close it either.
    2. A third KillTask shape. The other child killed an already-finished npm install. The frame came back as rawInput.variant: "KillTask", rawOutput.type: "KillTask", outcome: "already_exited", final title kill <id> (already_exited). extractXAiKilledBackgroundTasks matches on the kill_command_or_subagent title or a kill variant, so this shape is not recognized. @al0x99 saw (killed); already_exited is another variant to cover.
    3. Grok side is clean on 1.0.62. Driving grok agent --always-approve stdio with a minimal ACP client outside T3, five scenarios (read-only, echo, a >30 s command that auto-backgrounds, kill of a backgrounded command, kill of an already-exited task) all returned end_turn, emitted _x.ai/task_completed, and the process exited 0 on stdin close. Grok's unified.jsonl also logged shell.handle_prompt.done ok:true for both stuck children.
    4. task_cancel does not close the Grok session. After cancel the runs flipped to interrupted, but the two grok agent processes stayed alive and their logs show no shell.cancel.processing. Only the children that reached completed were closed by the idle session_close ~30 min later.

    Repro that should hit it every time (not run inside T3 by me, derived from the DB rows): delegate to a Grok child "run sleep 40 && echo done; when it is auto-backgrounded, wait on it with get_command_or_subagent_output, then reply and end your turn."

    #19938 addresses the pre-settlement task_completed drop, which is the path both of my children took. I haven't checked whether it also covers the call-… id and already_exited shapes above.
11. NotPellew commented on Oct 19, 2026

    

    NotPellew

    on Oct 19, 2026

    More actions

    Additional reproduction on 2026-10-19, using T3 Code desktop nightly 0.1.1-nightly.20261018.3302, commit d0918f1659f4c19172796c8d1910ca6e4bc18b51 (Linux).

    Setup and reproduction

    A parent thread delegates an implementation task through T3's delegate_task, with mode: "async", isolation: "worktree", provider instance grok, and selected model grok-build.

    Two hangs were observed in one session:

    • In the first occurrence, the delegated subagent thread committed its work, then went idle without posting its report. The task remained running until a follow-up message was steered into the thread. It then posted the report and the run terminated.
    • In the later occurrence, the delegated subagent thread finished its work, committed, and posted the full final assistant report. The timeline then had no further tool calls, reasoning, or messages until the user pressed Stop. T3 continued reporting status: running / workState: working, and the parent's completion notification never fired.

    This is an observed reproduction, not a deterministic minimal repro.

    Concrete timeline and independently checked Grok completion evidence

    All timestamps below are UTC on 2026-10-19.

    • 11:55:03Z: the full final assistant report appears in the T3 timeline. The last preceding actions were ordinary git status, git rev-parse, and git diff --stat tool calls.
    • 11:55:03Z: the Grok session's persisted update log records sessionUpdate: turn_completed, with stop_reason: end_turn, after the final assistant message update.
    • 11:55:03.737Z: the Grok session's event log records turn_ended, with outcome: completed. This is the last event in that log.
    • 11:59:39Z: after approximately 4 minutes 36 seconds with no additional timeline items, the user manually interrupts. T3 records run_interrupt_request ("Interrupt requested") and run_interrupt_result ("Run interrupted by user"). The task ends as interrupted, despite the completed work and final report.

    The T3 timeline and task-status observations above were verified by the parent orchestrator. The Grok session files were inspected read-only for this comment; no raw logs or task contents are attached. The persisted Grok updates establish that Grok recorded completion, but do not prove that the completion notification reached T3 over the ACP transport.

    Expected behavior and impact

    The delegated subagent run should become completed and notify its parent when the Grok turn finishes, without requiring a manual interrupt or follow-up message.

    In the same session, comparable delegated runs using opencode, antigravity, codex, and claudeAgent reached terminal states on their own.

    Workaround and limits

    A manual interrupt after the final report releases the stuck task but records it as interrupted. Steering a follow-up message released the first occurrence.

    No live repro was started for this report. I have not established which T3 projection rows, background-work hold, or completion-handling path caused these particular hangs. The Grok CLI version was not verified.
12. EgerDev commented on Oct 20, 2026

    

    EgerDev

    on Oct 20, 2026

    More actions

    Still reproduces on T3 Code 0.1.1-nightly.20261017.3272 (commit 13aa882c22d5) with Grok CLI 1.0.62, macOS 27 (Darwin 27.0.0, Apple M4, arm64).

    Setup: a claudeAgent / claude-opus-5-5 parent calls delegate_task (mode: "async") to grok / grok-4.6, with reasoningEffort: "xhigh" and full access.

    Result: 5 of 7 Grok children stayed running / working with summary: null after their final answer. The parent was never notified.

    • Two of them stayed that way for 7h40m, until task_cancel turned them into interrupted.
    • Three are still running hours later.

    In the same parent thread, children on Devin ACP (14+ tasks), Codex and Claude always settled.

    Each stuck child matches the deferred-finalize hold from the triage here. Grok logged turn_completed / end_turn at the same second as the final assistant_message. Yet the run, the provider turn and one or more command_execution rows stayed running, and no checkpoint item was ever written. The four Grok children that completed had no open background-shell rows.

    child (clientRequestId)	final answer = turn_completed (UTC)	rows still running	how Grok ended them (updates.jsonl)
    engine-check-…-E06-grok	07:07:03	2	(a) auto-backgrounded shell, task_id = call-…: task_completed exit 0 at 07:03:06; (b) UUID task stopped by KillTask at 07:04:48
    engine-check-…-E03-grok	07:54:24	1	auto-backgrounded shell, task_id = call-…: task_completed exit 0 at 06:33:54, 80 min before turn end. Two UUID background tasks in the same turn closed fine.
    engine-check-…-E02-grok	07:24:18	2	two UUID tasks: task_completed exit 0 at 07:19:09 and 07:19:16; one also hit by KillTask at 07:21:04 after it had exited
    cleanup-…-fix-ingest	21:37:13 (10-19)	3 (call-… ids)	not checked; rows changed from running to interrupted at task_cancel
    cleanup-…-fix-tooling	21:35:57 (10-19)	1 (call-… id)	not checked; same as above
    E04-grok, E05-grok, r1-D, r2-B-by-grok	—	0	completed and notified normally

    The triage query on one stuck child (E06) shows command_execution|running|2, with no subagent rows. The completed sibling (E05) has no running rows and has checkpoint|completed|1.

    Notes:

    • fix(provider-acp): retain Grok command completion before prompt settles #19938 (merged 2026-10-20 04:56 UTC) should cover E02. It is not in any nightly yet (the latest, 20261020.3386, was published at 01:46 UTC), so I couldn't test it.
    • E03 and E06(a) are the call-… task id shape that @letrandat and @wookiist describe. E06(b) and E02 are KillTask.
    • From the orchestrator's side, the worst part is that task_status looks exactly the same for a child that is still working and for one that finished an hour ago. Then task_cancel throws the finished result away as interrupted. A time limit on the hold once root end_turn has arrived would turn all of these shapes into a short delay instead of a permanent hang. So would a visible "held by background work" workState.
13. 3xcalibur-tech commented on Oct 20, 2026

    

    3xcalibur-tech

    on Oct 20, 2026

    Contributor

    More actions

    Fresh live verification on October 20, 2026, 13:23–13:26 UTC: I could not reproduce the stuck-running failure in five targeted scenarios on a build containing #19938. There is a separate result-delivery problem described below, so this is not an unconditional “everything is fixed.”

    Environment:

    • Installed macOS desktop 0.1.1-nightly.20261020.3411, embedded commit d36b7da2485bc889b320a285deea181f637d760a.
    • macOS 27.0.1, arm64; Grok 1.0.62 (1474f9cbdd91) [stable].
    • Real delegate_task calls through the running desktop server: mode: "async", isolation: "shared", grok / grok-4.7, reasoning low, full access.
    • Parent was Codex / GPT-6 Astra, not the Claude parent used in the original report.
    • Verified the installed app's build metadata, that merge commit e294a38a106acca87933d36789a4e0bd28a6e72a is an ancestor of its build commit, and that the shipped server bundle contains the retained command-completion implementation. These were tests of the installed server, not the source checkout.

    The recent reports above used builds from before #19938 merged at 2026-10-20 04:56:35 UTC. They establish the earlier failure but do not establish a regression after that fix.

    Scenarios actually exercised, one fresh Grok child each:

    1. Background sleep 5; echo T3_15888_EARLY_BG_DONE, then foreground sleep 15; echo T3_15888_FOREGROUND_DONE, without polling or killing the background command. Its completion arrived before root settlement. Command, provider turn and run all completed.
    2. Same ordering with an intentional background exit 7. The command correctly became failed, retained its output, and the provider turn/run completed.
    3. Foreground sleep 40; echo T3_15888_AUTO_BG_DONE with block_until_ms=30000, then get_command_or_subagent_output on the returned id. Verified actual auto-backgrounding with task_id == tool_call_id == call-… and a TaskOutput.Result for that id. The original command row and run completed.
    4. Background sleep 45; echo T3_15888_KILL_TARGET_DONE, then native kill_command_or_subagent on that task. Verified rawInput.variant: "KillTask", rawOutput.type: "KillTask", outcome killed. The command row and run completed.
    5. Background sleep 5; echo T3_15888_EXITED_BG_DONE, foreground sleep 12; echo T3_15888_WAIT_DONE, then native kill of the finished task. Verified the already_exited outcome. The command row and run completed.

    Grok's persisted native lifecycle timestamps and read-only T3 projections agree. Two examples, UTC:

    Early success
    13:24:06.557  Grok task_completed, exit_code=0
    13:24:21.245  Grok root turn_completed, end_turn
    13:24:21.246  T3 background command and provider turn completed
    13:24:21.325  T3 root run completed, checkpoint recorded
    
    Auto-backgrounded call-… id
    13:25:36.605  Grok task_backgrounded, task_id equals tool_call_id
    13:25:46.705  Grok task_completed, exit_code=0
    13:25:49.116  Grok root turn_completed, end_turn
    13:25:49.117  T3 original command and provider turn completed
    13:25:49.161  T3 root run completed, checkpoint recorded
    

    All five tasks returned status: "completed" / workState: "result_available". All root and continuation runs were terminal, with zero pending/running timeline items at the final check. Automatic completion notifications reached the parent for all five, before I read their terminal task_status. No Stop, task_cancel, follow-up steering or server restart was needed.

    Separate remaining problem: four of the five parent notifications and published task summaries said “Child task completed without an assistant result.” Every root run nevertheless has its requested final assistant reply in the T3 transcript. For those four tasks, the persisted subagent_result transfer points to run 2, an empty continuation, rather than the root run containing the reply. In the early-success case, a later latestTerminalSummary contains the expected reply while the published summary stays the placeholder. The early-failure case delivered the expected reply normally. This looks like a continuation/result-selection issue; it is distinct from a child remaining running forever or the parent never being notified.

    Conclusion: #19938's early command-completion fix works in this live setup, including the observed call-… and kill scenarios; the original hang did not reproduce. Result forwarding still needs attention. This was a small targeted sample, not a long-running stress test. Grok emitted a task_completed lifecycle notification in every case, including the killed task, so these passes do not prove that TaskOutput/KillTask-only fallback parsing works when that notification is absent. MultiResult, nested native subagents, Claude-parent and worktree-isolated variants were not tested.
14. t3dotgg

    closed this as completedin #22709on Oct 21, 2026
15. lukemaj

    added a commit that references this issue on Oct 29, 2026

    fix(server): carry upstream Grok fixes for stuck background commands …

    ...

    50ae1ba