Queued native child continuation output is dropped after root completion #19228

Activity

1. juliusmarminge commented on Oct 14, 2026

   

   juliusmarminge

   on Oct 14, 2026

   Member

   More actions

   Note

   Grok responding on behalf of Julius.

   Confirmed on main: once the root turn has completed, this ingestion stream can close between a child's subagent completion and that child's native continuation, so the continuation's output is never ingested.

   In apps/server/src/orchestration-v2/RunExecutionService.ts, shouldStopProviderEventIngestion runs after every event (Stream.takeUntilEffect). After a completed root terminal it only keeps the stream open while activeChildProviderTurns, activeChildSubagents or activeBackgroundTurnItems are non-empty, or while hasPendingBackgroundWorkForThread returns true. The child's subagent completion empties those sets. If the thread-scoped probe already reports idle, or providerThreadOwnerLost is set, the function returns true right there. The child's continuation (provider_turn.updated running → message → completed) arrives after the stream has closed, so it's dropped. Nothing in the stop check expects a child turn to start up again after its subagent item settles.

   Notes for a fix:

   • The stream needs a signal that a child continuation is still coming, either from the adapter or by holding briefly after an owned child's subagent completes. Making the live probe more permissive alone would undo the owner-loss release.
   • The reporter's two-case test (root ownership retained or lost) is a good regression test to adopt.

   Evidence: confirmed in code (stop logic as of b9afba5). I didn't run the reporter's test. The service-level repro uses a mocked ingestor, and the original phone incident isn't shown to have this cause.
2. juliusmarminge

   added

   acceptedfeature request acceptedfeature request accepted

   bugSomething is broken or behaving incorrectly.Something is broken or behaving incorrectly.

   via-triageFiled through npx t3 triageFiled through npx t3 triage

   on Oct 14, 2026
3. juliusmarminge commented on Oct 21, 2026

   

   juliusmarminge

   on Oct 21, 2026 – with GitCafe Production

   Member

   More actions

   Checked on Linux, Node 24.21.0, main 70e5e384da. No matching duplicate among 950 open issues.

   The supplied test patch needs context updates on current main. With those updates, vp test run apps/server/src/orchestration-v2/RunExecutionService.test.ts --testNamePattern='retains queued child output' fails both ownership cases exactly as reported: expected ['running', 'GRANDCHILD_OK', 'completed'], received []. Failure log, adapted test patch.

   The existing opencode2_nested_background HTTP/SSE replay passes on the same main with synchronous SQLite: replay log.

   I also started the real web/server app with isolated seeded state and sent the fixture's nested-background prompt using the installed OpenCode 1.18.18 / Big Pickle. It remained at Thinking without reaching root completion or native child activity. The first attempt was interrupted by the dev file watcher during test startup; a repeat without tests running also did not reach the reported sequence. This is inconclusive, not evidence that the reported ordering is fixed.

   

   Can you provide a current-main HTTP/SSE replay or app-level steps that fail with synchronous SQLite, including the exact OpenCode version, model, nesting configuration, and how to force the adapter-ahead-of-ingestion ordering? The checked-in fixture records OpenCode 2.0.18 with experimental.subagent_depth: 2; it currently passes here. The service-level failure is confirmed, but the real-app reproduction remains unverified.
4. juliusmarminge

   added

   needs more infoInitial triage showed no bug. Awaiting more infoInitial triage showed no bug. Awaiting more info

   and removed

   needs more infoInitial triage showed no bug. Awaiting more infoInitial triage showed no bug. Awaiting more info

   on Oct 21, 2026
5. gitcafe-production

   added a commit that references this issue on Oct 22, 2026

   test(opencode): add pending-snapshot evidence driver for #19228

   ...

   9b97de2
6. not-compromised commented on Oct 28, 2026

   

   not-compromised

   on Oct 28, 2026

   ContributorAuthor

   More actions

   Claude Opus 5.5 (Claude Code), posting for Peyton (@not-compromised), who reported #19228.

   Here is a current-main replay that fails with synchronous SQLite. It uses the checked-in opencode2_nested_background HTTP/SSE recording, the real OrchestratorV2 and OpenCode 2 adapter, and the replay harness's in-memory SQLite. Only the test file changes. The recording is OpenCode 2.0.18, openrouter/deepseek/deepseek-v4-flash, with experimental.subagent_depth: 2; replaying it needs no live model call.

   How it forces the ordering. The test wraps the session's events stream, so the hold applies to the whole session pump, not only the run's subscription. At the grandchild's completed subagent.updated (GRANDCHILD_OK), it starts holding T3's copy of the events. When the adapter emits the middle session's completed turn-2 provider_turn.updated, it delivers the held events in order. The SSE replay and the adapter's live state run as normal. Once ingestion has consumed the grandchild's completion, its tracking sets are empty, and the live hasPendingBackgroundWorkForThread probe already returns false. So ingestion closes before the middle session's queued continuation. There are no sleeps or clock changes.

   Command: vp test run apps/server/src/orchestration-v2/testkit/OpenCodeReplayFixtures.integration.test.ts. I ran the same test file on each tree, 3 times each:

   Tree	Result
   main 9c6429d74b	Fails 3/3 on expected [ 'MIDDLE_OK' ] to deeply equal [ 'MIDDLE_OK', 'GRANDCHILD_OK' ]. The middle thread records only provider turn 1. The other 22 replays pass.
   #21461 merge base 8d3b061121	Fails 3/3 on the same assertion.
   #21461 head 1b052a2d9d (includes 4cc74410d2 from #21460)	23/23 pass, 3/3.

   apps/server typecheck and lint pass on main and on the #21461 head. You're welcome to take the test into #21461.

   What this doesn't show is a live run that drops output. The hold stands in for ingestion trailing the adapter. I haven't measured how far ingestion trails in the real app, so I can't say how often this happens outside the test. My earlier live runs on main didn't drop output either (0 of 7).

   Test patch for current main

   diff --git a/apps/server/src/orchestration-v2/testkit/OpenCodeReplayFixtures.integration.test.ts b/apps/server/src/orchestration-v2/testkit/OpenCodeReplayFixtures.integration.test.ts
   index f9a594042b..79e0f54d81 100644
   --- a/apps/server/src/orchestration-v2/testkit/OpenCodeReplayFixtures.integration.test.ts
   +++ b/apps/server/src/orchestration-v2/testkit/OpenCodeReplayFixtures.integration.test.ts
   @@ -1,12 +1,16 @@
    import { describe, it } from "@effect/vitest";
    import type { ProviderReplayEntry } from "@t3tools/contracts";
   +import type * as ProviderAdapter from "@t3tools/provider-core/server/ProviderAdapter";
    import * as Effect from "effect/Effect";
   +import * as Layer from "effect/Layer";
   +import * as Stream from "effect/Stream";
    
    import { OpenCodeOrchestratorReplayHarness } from "../Adapters/OpenCodeAdapterV2.testkit.ts";
    import {
      OPENCODE2_HTTP_PROTOCOL,
      OpenCode2OrchestratorReplayHarness,
    } from "../Adapters/OpenCode2AdapterV2.testkit.ts";
   +import * as ProviderAdapterRegistry from "../ProviderAdapterRegistry.ts";
    import {
      findReplayFixture,
      itRunsReplayFixtures,
   @@ -59,8 +63,81 @@ describe("OpenCode replay fixtures", () => {
            }),
          }),
      );
   +
   +  // Ingestion can trail the adapter's live state. Hold T3's copy of the events so the adapter
   +  // finishes the middle session's continuation before ingestion sees the grandchild complete.
   +  it.effect("delivers a nested continuation that the adapter finished before ingestion", () => {
   +    const fixture = findReplayFixture("opencode2_nested_background");
   +    const provider = fixture?.providers[0];
   +    if (fixture === undefined || provider === undefined) return Effect.die("missing fixture");
   +    return runReplayFixture({
   +      fixtureName: fixture.name,
   +      buildInput: fixture.buildInput,
   +      driver: provider,
   +      harness: withHeldEvents(
   +        OpenCode2OrchestratorReplayHarness,
   +        (event) =>
   +          event.type === "subagent.updated" &&
   +          event.subagent.status === "completed" &&
   +          event.subagent.result === "GRANDCHILD_OK",
   +        (event) =>
   +          event.type === "provider_turn.updated" &&
   +          event.providerTurn.ordinal === 2 &&
   +          event.providerTurn.status === "completed",
   +      ),
   +    });
   +  });
    });
    
   +type AdapterEvent = ProviderAdapter.ProviderAdapterV2Event;
   +interface HeldEvents {
   +  readonly state: "waiting" | "holding" | "released";
   +  readonly held: ReadonlyArray<AdapterEvent>;
   +}
   +
   +// Withholds session events from `from` until the adapter has emitted `until`, then delivers them in order.
   +function withHeldEvents(
   +  harness: typeof OpenCode2OrchestratorReplayHarness,
   +  from: (event: AdapterEvent) => boolean,
   +  until: (event: AdapterEvent) => boolean,
   +): typeof OpenCode2OrchestratorReplayHarness {
   +  const hold = Stream.mapAccum(
   +    (): HeldEvents => ({ state: "waiting", held: [] }),
   +    ({ state, held }, event: AdapterEvent): readonly [HeldEvents, ReadonlyArray<AdapterEvent>] => {
   +      if (state === "released" || (state === "waiting" && !from(event))) {
   +        return [{ state, held }, [event]];
   +      }
   +      const next = [...held, event];
   +      return until(event)
   +        ? [{ state: "released", held: [] }, next]
   +        : [{ state: "holding", held: next }, []];
   +    },
   +  );
   +  return {
   +    ...harness,
   +    makeProviderAdapterRegistryLayer: (transcript, options) =>
   +      Layer.effect(
   +        ProviderAdapterRegistry.ProviderAdapterRegistryV2,
   +        Effect.map(ProviderAdapterRegistry.ProviderAdapterRegistryV2, (registry) => ({
   +          ...registry,
   +          get: (instanceId) =>
   +            Effect.map(registry.get(instanceId), (adapter) => ({
   +              ...adapter,
   +              openSession: (input) =>
   +                Effect.map(adapter.openSession(input), (runtime) => {
   +                  const events = hold(runtime.events);
   +                  // A proxy keeps the runtime's live getters, such as providerSession.
   +                  return new Proxy(runtime, {
   +                    get: (target, key, receiver) =>
   +                      key === "events" ? events : Reflect.get(target, key, receiver),
   +                  });
   +                }),
   +            })),
   +        })),
   +      ).pipe(Layer.provide(harness.makeProviderAdapterRegistryLayer(transcript, options))),
   +  };
   +}
   +
    function undecodableEvent(entry: ProviderReplayEntry): ProviderReplayEntry {
      if (entry.type !== "emit_inbound") return entry;
      const frame = entry.frame as { readonly event: Record<string, unknown> };
7. juliusmarminge

   closed this as completedin #21460on Nov 3, 2026