v2: concurrent application graphs can send unmatched tool calls for one Session #43648

Description

OpenCode V2 can run overlapping model/tool pipelines for the same durable Session when independent application graphs share its database. One pipeline can then construct an OpenAI Responses request containing another pipeline's still-running local tool call without the corresponding tool result. The provider correctly rejects that malformed history with:

No tool output found for function call call_<id>.

The tool itself does not necessarily fail. In the clearest occurrence, the original shell command continued running and completed normally after the concurrent provider request had already been rejected.

Observed timeline

The following timestamps came from a sanitized Session export:

The persisted shell state eventually contained both ran and completed timestamps and successful output. A different assistant message recorded the provider error with finish: "error" before the shell completed.

A second occurrence happened around a managed-service restart. Two assistant messages were created about 90 ms apart and both launched tools. One tool result was durable by the time the provider error was observed, which is consistent with the rejected request having been materialized from an earlier history snapshot.

Why this does not appear to be a normal same-process steer

At the observed V2 revision, one SessionRunCoordinator serializes execution per Session within an application graph:

• packages/core/src/session/run-coordinator.ts joins concurrent run() calls and turns active wake() calls into a pending doorbell.
• packages/core/src/session/execution.ts routes production drains through that coordinator.
• packages/core/src/session/runner/llm.ts owns every local tool fiber and awaits all of them before advancing to another model step.

Therefore, one normal coordinator cannot start the second assistant 183 ms after the first while still owning the first assistant's shell for another 27 seconds. The timing strongly indicates that at least two independent SessionExecution/SessionRunner graphs were operating on the same Session.

The supplied evidence does not prove exactly how the second graph was started. Possible sources include overlapping managed services on different endpoints, a managed service plus a standalone/foreground server, an embedded server graph, or independently configured servers sharing OPENCODE_DB.

How the malformed request is produced

The likely sequence is:

1. Graph B starts a model request and loads history before graph A's shell call is visible.
2. Graph A publishes call_d197... as running and starts the long shell.
3. Graph B receives its response, creates the second assistant, and completes its own read tool.
4. Graph B advances to its continuation and reloads current durable history.
5. That history now contains graph A's shell in running state but no shell result.
6. packages/core/src/session/runner/to-llm-message.ts emits a tool-call part for a running local tool, but toolResult() emits a result only for completed or error states.
7. packages/ai/src/protocols/open-responses.ts lowers the call to function_call; it can emit function_call_output only when a tool-result message exists.
8. The provider rejects the unmatched function call.
9. Graph A later completes the shell and updates durable state, but that cannot change graph B's already-built or already-sent request.

This also explains why a Session can look valid when inspected after the incident even though the provider received malformed history earlier.

Cross-graph ownership gap

The durable execution claim does not enforce ownership. SessionStore.claim() updates time_suspended only when it is null, but the result is not used as a compare-and-acquire decision. The claim has no owner ID, lease, or fencing token. A second graph can affect zero rows and still begin its drain.

Restart recovery in packages/core/src/session/execution/restart.ts excludes Sessions found in the current graph's process-local active set. It cannot detect that another process or graph still owns a claimed Session, so it can treat a live claim as orphaned and resume it.

Managed-service registration is also a lease rather than an exclusion lock. Same-host/same-port contenders are normally excluded by the TCP bind before the application graph is built. If contenders use different endpoints, however, both can bind. The newer process can replace the shared registration while the previous process remains active until its five-second registration ownership check notices the replacement.

The stale-tool sweep does not make cross-graph execution safe. It runs once at drain entry. Graph B can pass that sweep before graph A publishes its tool, then encounter the running tool on a later continuation. If the sweep instead sees a tool still owned by graph A, marking it stale would incorrectly abort live work.

Expected behavior

Only one application graph should be allowed to drive a durable Session at a time.

If multiple graphs sharing a database are unsupported, OpenCode should prevent the second graph from building/running Session recovery. If they are supported, Session execution needs durable ownership and fencing.

Request assembly must never send an OpenAI-compatible provider an assistant function call without its required function output. Encountering another execution's unresolved local tool should delay or fail the conflicting execution rather than constructing malformed provider history.

Current V2 status

The occurrence was observed on opencode2 v0.0.0-dev-17604, corresponding to V2 commit 643eed300dc34b15fd8742243298fa882d6d9199.

The relevant failure class is still present at V2 commit 838d74751412e0f9a38a2e3283fd867a36d76900:

• Session coordination remains application/process local.
• Durable claims remain non-exclusive recovery markers.
• Restart recovery still checks only local active execution.
• Running local tools are still lowered as calls without results.
• Managed registration is still replacement followed by a five-second ownership poll.

Recent service lifecycle changes make normal registered-process replacement safer, but they do not provide per-database or per-Session fencing across independent graphs.

Suggested remediation

First decide and enforce the supported ownership boundary:

• If one application graph per database is required, acquire a database-scoped service lock before building the graph or running restart recovery, and hold it for the graph lifetime. Endpoint-specific TCP binding is not sufficient.
• If multiple graphs per database are supported, add durable per-Session ownership with an owner identity and fencing token. Every continuation and terminal write must verify the current fence.
• Treat unresolved local tool calls from another execution as an ownership conflict. Do not omit the result and send malformed provider history.
• Run stale-tool recovery only after ownership of the Session has been established.

Suggested regression coverage

Add a test using two complete application graphs over one temporary database and a fake provider:

1. Start graph B and gate its first provider response after it has passed drain entry and loaded history.
2. Start graph A for the same Session and have its provider request a deliberately blocked local tool.
3. Release graph B with a fast local tool call and let that tool complete.
4. Make graph B's next provider call validate that every local tool call has a matching result.
5. Assert that graph B cannot send a request while graph A owns the unresolved tool.
6. Release graph A's tool and assert exactly one valid continuation with one result per call.

Also add a managed-service integration test in which two endpoints share one database and registration location. Assert that only one graph may perform restart recovery or drain a claimed Session, not merely that one registration eventually wins.

Plugins

No response

OpenCode version

Observed on opencode2 v0.0.0-dev-17604 / V2 commit 643eed300dc34b15fd8742243298fa882d6d9199. Source audit confirms the failure class remains at V2 commit 838d74751412e0f9a38a2e3283fd867a36d76900.

Steps to reproduce

The observed user-level sequence was:

1. Run V2 through the managed background service with an OpenAI Responses-compatible model.
2. Ask the model to perform work that starts a long-running built-in shell tool.
3. While the shell is visibly running, submit a steering/correction message.
4. Observe a second assistant/tool pipeline begin before the original shell settles.
5. Observe a later provider request fail with No tool output found for function call call_<id>.
6. Export the Session and compare assistant creation, tool start/completion, and provider-error timestamps.

The steering message alone should not create overlap in one application graph. A deterministic reproduction should use the two-graph fake-provider test described above, because the remaining unknown is how the second graph was present in the live occurrence.

Screenshot and/or share link

No response

Operating System

No response

Terminal

No response