Repository navigation
Gemini sets error_code for a MAX_TOKENS response with content only when streaming #7973
Description
🔴 Required Information
Describe the Bug:
Gemini reports the same MAX_TOKENS response differently depending on the streaming mode. When the model returns text and stops with MAX_TOKENS:
- Without streaming,
LlmResponse.createkeeps the content and leaveserror_codeunset (llm_response.py#L246-L258). - With SSE,
StreamingResponseAggregator.close()setserror_codefor any finish reason other thanSTOP, content or not (streaming_utils.py#L577-L581).
The final event has the same text and finish_reason either way. Only error_code and error_message differ.
Each side is pinned by a test: test_create_non_stop_finish_with_parts_keeps_content_without_error in test_llm_response.py, and test_generate_content_async_stream_with_thought_and_text_error_handling in test_google_llm.py. So I'm not sure which one is intended. Code that reads error_code behaves differently by streaming mode:
ReflectAndRetryModelPlugin(on_model_errors=[types.FinishReason.MAX_TOKENS])only retries streamed turns, since_check_for_model_errorreturns early whenerror_codeis unset (_reflect_retry_model_plugin.py#L108-L112).- The A2A executor ends the task with an error status only when the turn was streamed (a2a_agent_executor_impl.py#L219-L220).
Steps to Reproduce:
- Check out
main(7662758) and runuv sync --extra test. - Save the script below as
repro.py. It fakes the genai client, so no API key is needed. - Run
uv run python repro.py.
Expected Behavior:
The same model response gives the same error_code in both streaming modes.
Observed Behavior:
NONE: finish_reason=MAX_TOKENS error_code=None text='The answer is forty'
SSE: finish_reason=MAX_TOKENS error_code=MAX_TOKENS text='The answer is forty'
Environment Details:
- ADK Library Version (pip show google-adk): 2.13.0 (main at 7662758)
- Desktop OS: macOS
- Python Version (python -V): 3.13.13
Model Information:
- Are you using LiteLLM: No
- Which model is being used: gemini-2.5-flash (client faked in the repro)
🟡 Optional Information
Additional Context:
Related but different: #7660 (no content, other model), #7693 (LiteLlm), and #7455 (field ownership when a callback replaces a response).
Minimal Reproduction Code:
import asyncio
from google.adk.agents import LlmAgent
from google.adk.agents.run_config import RunConfig, StreamingMode
from google.adk.models.google_llm import Gemini
from google.adk.runners import InMemoryRunner
from google.genai import types
def chunk(text, finish=None):
return types.GenerateContentResponse(candidates=[types.Candidate(
content=types.Content(role="model", parts=[types.Part(text=text)]),
finish_reason=finish,
finish_message="Maximum tokens reached" if finish else None,
)])
class FakeModels:
async def generate_content(self, **kw):
return chunk("The answer is forty", types.FinishReason.MAX_TOKENS)
async def generate_content_stream(self, **kw):
async def gen():
yield chunk("The answer ")
yield chunk("is forty", types.FinishReason.MAX_TOKENS)
return gen()
class FakeClient:
vertexai = False
class aio:
models = FakeModels()
async def run(mode):
llm = Gemini(model="gemini-2.5-flash")
llm.__dict__["api_client"] = FakeClient()
runner = InMemoryRunner(agent=LlmAgent(name="a", model=llm), app_name="app")
session = await runner.session_service.create_session(app_name="app", user_id="u")
events = [
e async for e in runner.run_async(
user_id="u",
session_id=session.id,
new_message=types.Content(role="user", parts=[types.Part(text="hi")]),
run_config=RunConfig(streaming_mode=mode),
)
if not e.partial
]
final = events[-1]
print(f"{mode.name}: finish_reason={final.finish_reason.name} "
f"error_code={final.error_code} text={final.content.parts[0].text!r}")
asyncio.run(run(StreamingMode.NONE))
asyncio.run(run(StreamingMode.SSE))How often has this issue occurred?:
- Always (100%)
Activity
its-amann commentedon Nov 3, 2026
Hi! I'd like to work on this. @GWeale, could you assign it to me? I'll follow up with a PR that includes a fix and tests.
r14dd commentedon Nov 3, 2026
Thanks for offering, but I reported this and I'm planning to fix it myself. Both behaviours are pinned by tests, so I'm waiting for a maintainer to say which one is intended before I open a PR.
surajksharma07 commentedon Nov 4, 2026
@r14dd Reproduced this on 2.13.0 too: SSE sets error_code=MAX_TOKENS while non-streaming keeps the text with no error.
As a workaround you could try an after_model_callback that clears error_code/error_message when the response still has content parts and error_code == finish_reason. That made both modes match in my local test, and an empty MAX_TOKENS reply still came back as an error.
My take is that streaming should follow LlmResponse.create, but ReflectAndRetryModelPlugin would then need to check finish_reason instead of error_code. Flagged it so a maintainer can confirm the intended behaviour.
Could you check whether that approach works with your A2A/retry setup and test it thoroughly before you open the PR?
r14dd commentedon Nov 4, 2026
Thanks for reproducing it. I tried your direction on main (
70e1cbd): inStreamingResponseAggregator.close(), clearerror_code/error_messagewhen the aggregated content has parts, the same ruleLlmResponse.createuses. I ran the same MAX_TOKENS-with-text response throughInMemoryRunnerin both modes.Two things came up.
Retry plugin. With
ReflectAndRetryModelPlugin(on_model_errors=[MAX_TOKENS]), SSE on main never stops: every clean partial resets the retry count, so it runs to the 500 llm call limit. That is ReflectAndRetryModelPlugin counts one streamed model error twice #7872, and the partial early return in fix(plugins): count a streamed model error once in ReflectAndRetryModelPlugin #7873 fixes it. With fix(plugins): count a streamed model error once in ReflectAndRetryModelPlugin #7873 applied I tried both plugin options:finish_reasononly, as you suggested: 3 calls in both modes. Non-streaming changes from 1 call to 3, andtest_check_for_model_error_requires_error_codefails, since it pins that a matching finish reason without an error code is not retried.error_code): 1 call in both modes, no plugin test fails.So the streaming change alone makes both modes agree. Changing the plugin as well would reverse a choice its tests pin on purpose.
Other finish reasons. The
createrule clears the error for every non-STOP reason once there is content, not only MAX_TOKENS.test_close_with_errorpins that a stream ending in RECITATION with text keepserror_code=RECITATION. SAFETY behaves the same way. With the change, SSE callers would see those only infinish_reason, as non-streaming callers already do. Three Gemini tests intest_google_llm.pypin the SSE side too (..._aggregated_content_regardless_of_finish_reason,..._with_thought_and_text_error_handling,..._error_info_set_for_non_stop_finish_reason).An empty MAX_TOKENS reply still comes back as an error in both modes, since the error is only cleared when there is content.
My plan, if a maintainer agrees: the streaming change only, plugin untouched, and the four pinned tests updated to the
createbehaviour. The other option is to scope it to MAX_TOKENS, which leaves SAFETY/RECITATION as they are in SSE. I'll wait for a maintainer to pick before opening the PR.