[Microsoft.Extensions.AI.OpenAI] Intermittent 400 previous_response_not_found when chaining Responses API turns with function invocation

[Licantrop0]

Description

When using the OpenAI Responses API through AsIChatClient() in a multi-turn console app with function/tool invocation, requests intermittently fail with a 400 from the service:

Previous response with id 'resp_...' not found.

("code": "previous_response_not_found")

The failure is intermittent — the same conversation can run for several turns, then a turn fails. Once it fails, retrying the same user turn usually succeeds, which suggests a stale/never-persisted response id is being sent as previous_response_id rather than a permanently corrupt conversation state.

Microsoft.Extensions.AI.OpenAI is what maps ConversationId onto previous_response_id for the Responses client, so filing here rather than against openai/openai-dotnet.

Versions

Package	Version
Microsoft.Extensions.AI	10.9.0
Microsoft.Extensions.AI.OpenAI	10.9.0
OpenAI	2.12.0
ModelContextProtocol	2.2.0

.NET 10, Windows. Endpoint is Azure OpenAI's OpenAI-compatible /openai/v1/ surface with BearerTokenPolicy.

Repro shape

var clientOptions = new OpenAIClientOptions
{
    Endpoint = new Uri($"{azureOpenAIEndpoint.TrimEnd('/')}/openai/v1/"),
    NetworkTimeout = TimeSpan.FromMinutes(5)
};

var openAIClient = new OpenAIClient(tokenPolicy, clientOptions);
var innerClient = openAIClient.GetResponsesClient().AsIChatClient();

var chatClient = new ChatClientBuilder(innerClient)
    .UseFunctionInvocation()
    .Build();

var chatOptions = new ChatOptions
{
    ModelId = model,
    AllowMultipleToolCalls = true,
    ToolMode = ChatToolMode.Auto,
    Tools = [.. mcpTools]   // tools obtained from MCP servers
};

IList<ChatMessage> chatHistory = [new ChatMessage(ChatRole.System, systemPrompt)];

// per user turn:
chatHistory.Add(new ChatMessage(ChatRole.User, userMessage));

List<ChatResponseUpdate> receivedUpdates = [];
await foreach (var update in chatClient.GetStreamingResponseAsync(chatHistory, chatOptions))
{
    receivedUpdates.Add(update);
    // ... render text / reasoning / function call + result content ...
}

chatHistory.AddMessages(receivedUpdates);

The app keeps a single IList<ChatMessage> history and a single shared ChatOptions instance across turns, appends streamed updates back into history via AddMessages, and never sets ChatOptions.ConversationId itself. Tools are MCP tools, so most turns include at least one function call round trip inside UseFunctionInvocation.

What I ruled out

The original app also had .UseDistributedCache(...) and .UseChatReducer(new SummarizingChatReducer(...)) in the pipeline, so my first assumption was that one of those was replaying or rewriting history in a way that resurrected a stale conversation id. That turned out not to be it:

• Reproduces with and without UseDistributedCache.
• Reproduces with and without UseChatReducer / SummarizingChatReducer.
• Reproduces with local (stdio) MCP servers and with remote (HTTP) MCP servers, so it isn't specific to one transport or tool set.

With all of the optional middleware removed the failure still occurs, which is why I believe it lives in the Responses IChatClient response-chaining path (or in how ConversationId is round-tripped through streamed updates and AddMessages) rather than in application middleware.

Expected

Either:

1. A conversation id that the service no longer recognizes should not be sent as previous_response_id — the client should fall back to sending the full input history instead of a dangling chain; or
2. If chaining is expected to be the caller's responsibility here, it would help a lot to have documented guidance on when ConversationId is safe to carry across turns when UseFunctionInvocation performs intermediate round trips, and how a consumer is meant to detect and recover from previous_response_not_found.

Actual

An intermittent 400 previous_response_not_found surfaces to the caller as a ClientResultException, terminating the turn.

Notes

Happy to instrument and capture request payloads (previous_response_id vs. submitted input items) for the failing turn if that would help pin it down — let me know what would be most useful.

[akirayamamoto]

We hit this on the same package versions and the same Azure /openai/v1/ surface, streaming with function invocation. The investigation and all the measurements below were done by Claude against our own deployment. The data points somewhere slightly different from the current framing, so it seemed worth writing up.

The response is persisted, it just is not readable yet

The issue body suggests a stale or never-persisted response id. It does get persisted. Polling GET /openai/v1/responses/{id} straight after the parent completes returns 404 for a while, then 200. Across 79 sampled responses, every single one eventually became retrievable. Worst case was 32.7 seconds.

So the id being sent is correct. The service just cannot resolve it yet.

stream: true on the parent is the trigger

This is the part I have not seen reported anywhere. I ran a randomised-order probe measuring the time from "status": "completed" to a GET returning 200, rotating cell order within each round to control for ordering effects:

Parent request	n	median	max	share over 7s
streamed, with tools	13	7.0s	30.1s	53%
streamed, no tools	14	5.5s	21.5s	50%
streamed, no tools, explicit store: true	13	2.8s	15.7s	38%
streamed, no tools, gpt-5-mini	13	3.4s	32.7s	30%
non-streamed, with tools	13	1.0s	3.9s	0%
non-streamed, no tools	13	0.9s	11.6s	15%

The last column uses 7 seconds because that was the fastest rejection I observed, so anything slower than that is roughly the danger zone.

Aggregated over the gpt-5.6 rows: streamed parents take a median 5.1s to become readable, non-streamed 1.0s. That is n=40 against n=26, on a GlobalStandard deployment in East US 2. The gpt-5-mini deployment on the same resource is affected too, just less.

Tool calls make it worse on top of streaming, which fits the reports here being about tool-calling loops. Nothing about function invocation is required though. Streamed with no tools at all still fails half the time.

The reason it looks like a client bug is timing. A tool continuation fires within a millisecond or two of the parent completing, which lands inside the window almost every time. A human typing the next turn has already waited it out.

This reconciles the conflicting retry advice

semantic-kernel#13128 was explicitly non-streaming, and a 50/100/200ms backoff fixed it there. That matches the 1.0s median in the non-streamed rows. At a streamed median of 5.1s, a millisecond-scale backoff does nothing, which is presumably why the same advice has not worked for people on this thread.

The service also spends real time before giving up. Rejected calls came back after 7.0 to 15.8 seconds, so each failed attempt is expensive and a naive retry loop gets slow quickly.

Standalone repro, no Microsoft.Extensions.AI involved

I reduced it to two raw HTTP calls with no SDK in the path: create a streamed response, then immediately continue from it with previous_response_id. Python standard library only, credentials from environment variables.

REPRODUCED: 4/15 continuations rejected with previous_response_not_found
  rejected calls took 7.0-15.8s
  accepted calls took 2.1-22.3s

Every parent reported status=completed store=True, and every rejection had a client-side gap under 2ms.

Since it reproduces without this library, I do not think the fix belongs here. Microsoft.Extensions.AI.OpenAI is doing the right thing with the id it is given. Worth redirecting to Azure, though this repo is where people keep landing, so leaving the measurements here in case they save someone else the week.

Workaround, if anyone needs one now

Setting StoredOutputEnabled = false makes OpenAIResponsesChatClient stop surfacing a ConversationId, so the agent resends conversation history instead of chaining by id:

var options = new ChatOptions
{
    RawRepresentationFactory = _ => new CreateResponseOptions { StoredOutputEnabled = false }
};

That took us from 6 rejections in 10 continuations to 0 in 24. Three things I checked before adopting it, since none of them were obvious:

• Reasoning context survives. Resent reasoning items carry encrypted_content with a stable item id across hops.
• Prompt caching still works. A second hop cached 5010 of 5013 input tokens.
• Token cost went up about 6%, because the system prompt dominates and is resent either way.

What you give up is server-side retrieval of responses, background mode, and resuming a stream after a disconnect.

Happy to share the probe scripts or the raw samples if they are useful.

[akirayamamoto]

Adding the repro itself, since "two raw HTTP calls" is easier to check than to take my word for. Also one correction to what I wrote above.

Create a streamed parent and read the id off the response.completed event:

curl -sS "$ENDPOINT/openai/v1/responses" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"model":"'"$DEPLOYMENT"'","input":"Reply with exactly: pong","stream":true}' \
  | grep '"response.completed"' | sed 's/^data: //' \
  | python3 -c 'import json,sys; r=json.load(sys.stdin)["response"]; print(r["id"], r["status"], r["store"])'

That prints resp_0a1b... completed True. Immediately continue from it:

curl -sS -w '\n%{http_code} in %{time_total}s\n' "$ENDPOINT/openai/v1/responses" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"model":"'"$DEPLOYMENT"'","input":"Reply with exactly: pong","previous_response_id":"'"$PARENT_ID"'","stream":false}'

$TOKEN is an Entra token for https://cognitiveservices.azure.com/.default, from a principal with the Cognitive Services OpenAI User role.

The correction: a rejection is only half of it

I framed this above as a rejection rate, which undersells it. Looping those two calls today gave 5 rejections in 12, but the accepted ones are not free either:

REPRODUCED: 5/12 continuations rejected with previous_response_not_found
  rejected calls took 7.1-10.6s
  accepted calls took 2.2-48.1s

A 48 second continuation that returns 200 is the same wait as a rejection, just resolved in your favour. The service appears to block on the lookup until either the record appears or it gives up, and how long it is willing to wait varies. An earlier batch of five ran clean but took 6, 12, 18 and 22 seconds.

So the symptom is not only intermittent 400s. It is also multi-second latency on turns that succeed, which is easy to misread as the model being slow. If anyone here has been seeing unexplained long turns alongside the 400s, that is likely the same cause.

The rate moves around with load. Across runs today I saw anywhere from 0 in 5 to 5 in 12, so a single clean run does not mean much either way.

Two knobs worth trying

• Run the second call by hand a minute later and it succeeds every time. The window is what matters, not the id.
• Change "stream":true to false on the first call and both the rejections and the long waits mostly disappear. That is the whole finding in one flag.

Happy to attach the looping script if that is more useful than the two calls.

[arnabbiswas1]

I am hitting the exact same issue.

I have a client which uses raw Azure OpenAI Response API with tool/function calling (streaming). On 24th August 2026 around 9:30 UTC (It could be even before), I started observing the following error for the region East US 2 (model: gpt-5.2-2025-12-11-global-standard). It was sporadic in nature. Most of the times it was returning the error while sometimes it was working.

    Previous response with id 'resp_0fbbbcd970100556006a8c12818efc8196aee2019467f01855' not found.", 'type': 'invalid_request_error', 'param': 'previous_response_id', 'code': 'previous_response_not_found'

However, when switched to East US or North Central US (same model), the problem vanished.

Update: 25 August 2026 4:47 am, the frequency of occurrence in East US 2 (model: gpt-5.2-2025-12-11-global-standard) has reduced. But, it's still there.