Skip to content

Aborting during parallel tool calls records tools that already finished as "aborted" and drops their state changes #7958

Open

Description

@AkshayShah03

Describe the bug

When a model response contains several function calls, ADK runs them in parallel and records their results in one merged function-response event once all of them have finished. If the invocation is aborted while one call is still running (abort_signal.set(), a client disconnect on /run or /run_sse, or cancelling the task that drives run_async), the calls that already finished are sealed the same way as the unfinished ones: {"error": "Invocation was aborted by client."}. Their real results and their state_delta are dropped.

So the model is told that a tool which really ran did not. For a side-effecting tool this is the case the abort guide warns about: if the user says "go on" or "try again", the model calls it a second time.

To Reproduce

import asyncio

from google.adk.agents import LlmAgent
from google.adk.models.base_llm import BaseLlm
from google.adk.models.llm_response import LlmResponse
from google.adk.runners import InMemoryRunner
from google.adk.tools.tool_context import ToolContext
from google.genai import types

sent = []
slow_started = asyncio.Event()


def send_email(to: str, tool_context: ToolContext) -> dict:
  """Sends an email."""
  sent.append(to)
  tool_context.state["emails_sent"] = len(sent)
  return {"status": "sent", "to": to}


async def lookup_order(order_id: str) -> dict:
  """Looks up an order (slow)."""
  slow_started.set()
  await asyncio.sleep(30)
  return {"order_id": order_id}


class ScriptedModel(BaseLlm):
  model: str = "scripted"

  async def generate_content_async(self, llm_request, stream=False):
    yield LlmResponse(
        content=types.ModelContent(
            parts=[
                types.Part.from_function_call(
                    name="send_email", args={"to": "a@example.com"}
                ),
                types.Part.from_function_call(
                    name="lookup_order", args={"order_id": "42"}
                ),
            ]
        )
    )


async def main():
  agent = LlmAgent(
      name="agent", model=ScriptedModel(), tools=[send_email, lookup_order]
  )
  runner = InMemoryRunner(agent=agent)
  session = await runner.session_service.create_session(
      app_name=runner.app_name, user_id="u"
  )
  abort_signal = asyncio.Event()

  async def run():
    async for _ in runner.run_async(
        user_id="u",
        session_id=session.id,
        new_message=types.UserContent("email a@example.com and check order 42"),
        abort_signal=abort_signal,
    ):
      pass

  task = asyncio.create_task(run())
  await slow_started.wait()
  abort_signal.set()  # e.g. the user presses stop
  await task

  session = await runner.session_service.get_session(
      app_name=runner.app_name, user_id="u", session_id=session.id
  )
  print("send_email actually ran:", sent)
  print("state emails_sent:", session.state.get("emails_sent"))
  for event in session.events:
    for response in event.get_function_responses():
      print("recorded response:", response.name, response.response)


asyncio.run(main())

Output on main (73ae69e):

send_email actually ran: ['a@example.com']
state emails_sent: None
recorded response: send_email {'error': 'Invocation was aborted by client.'}
recorded response: lookup_order {'error': 'Invocation was aborted by client.'}

The same happens on the commit before abd8f0f, so it isn't from the recent teardown change. That change just makes plain task.cancel() reach the same path.

Expected behavior

send_email's real response ({"status": "sent", ...}) and its state_delta are recorded. Only lookup_order, which was still running, is sealed as aborted.

Root cause

_gather_or_cancel in src/google/adk/flows/llm_flows/tools/_batch_executor.py cancels the unfinished tasks and re-raises on cancellation. The results of tasks that already completed are discarded, since the merged response event is only built from the full list. _build_abort_events then finds every call in the batch without a response and seals all of them.

Possible fix

On cancellation, keep the response events of tasks that finished. For example, append a merged response event for just those calls, or hand them to the abort sealing so it uses them instead of the synthetic error. _build_abort_events already seals only calls that have no response, so it would then cover just the unfinished ones.

Question

Is recording finished results on abort the behaviour you want, or is it intended that an aborted batch is treated as a whole? If it's the former, I'm happy to open a PR with a regression test.

Desktop (please complete the following information):

  • OS: macOS
  • Python version: 3.12
  • ADK version: main (73ae69e)

Model Information:

  • Not model-specific: a scripted BaseLlm.

Activity

  1. its-amann commented on Nov 3, 2026

    @its-amann

    Hi @surajksharma07, are you working on this one? If you're busy I'd be happy to pick it up.

  2. AkshayShah03 commented on Nov 3, 2026

    @AkshayShah03
    ContributorAuthor

    I'm happy to take this if it's useful — I have the repro and the root cause in _gather_or_cancel, and can open a PR with a regression test once you confirm that keeping finished results is the behaviour you want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions