Python: [Bug]: AG-UI approval resume is consumed (and the approved tool executed) before the run can fail — a post-consume failure makes the user's answer unrecoverable, and the retry is indistinguishable from a never-pending id

[antsok]

Description

1. Problem

A tool-approval resume is single-use by design: when it validates, the pending approval entry is consumed to prevent replay, and the approved tool is executed — both at request ingress, before the agent (and therefore the model) runs:

# _agent_run.py:2068 — ingress, before the agent is invoked
resolved_approval_results = await _resolve_approval_responses(
    messages, tools_for_execution, agent, run_kwargs, pending_approvals, approval_thread_id
)

# _agent_run.py:1285-1292 — inside _resolve_approval_responses
# Valid — consume entry to prevent replay
_consume_pending_approval_entry(...)

# _agent_run.py:1333 — and the approved tool executes right there
approved_function_result_groups, _ = await _try_execute_function_call_groups(...)

Nothing records that the consume (or the execution) happened. So any failure after this point — a provider 4xx/5xx on the model call that follows, a transport drop mid-stream, a dying replica — burns the resume:

POST #1  → entry consumed → tool executed → …model call fails… → dead stream
POST #2  → entry gone     → APPROVAL_RESUME_NOT_FOUND

The retry is answered by _canonical_approval_resume_messages with APPROVAL_RESUME_NOT_FOUND (_agent_run.py:982/:1001/:1036) — the same code a never-pending interrupt id gets. Nothing in the system can distinguish "this resume was already applied", "this resume was consumed but never applied", and "this interrupt never existed". Two consequences:

• The user's answer is unrecoverable. The resume carries user data — the approval decision and any edited arguments (on hosts that route clarifying-question answers through approval-gated tools, the answers themselves ride the resume). After a post-consume failure there is no way to re-deliver it: the same resume is rejected forever, and the pending interrupt it belonged to is gone.
• The failure mode punishes exactly the clients that behave well. A client that retries after a dropped stream (reconnect logic, an auto-continue) hits NOT_FOUND on every retry. In the failure-half repro below, the approved tool's side effect already happened — and the retry still reports that the question was never pending.

A smaller aggravator on the same path: the NOT_FOUND branch can also clear the thread's persisted tool-approval middleware state (_agent_run.py:1965-1972), so a duplicate of a successful resume does not just get a wrong error — it can wipe approval state a live thread still needs.

To be clear: consuming on validation is the right call for replay protection (a resume must be single-use authority). The defect is that it is spent on the optimistic path with no outcome memory and no rollback, so the failure and duplicate cases collapse into the never-pending case.

2. Observed impact (deployed)

Hit in a deployed application on 2026-07-30 (Azure Container Apps): a provider returned 400 on the model call of every approval-resume run (that provider defect is reported separately — #7452). Every answered approval card then produced this sequence — entry consumed at ingress, run failed on the model call, every retry APPROVAL_RESUME_NOT_FOUND — so every answer the user gave was lost, and the app could only degrade to "please resend your answer as a new message", losing the interrupt and its edited arguments. The same shape occurs with no provider defect at all: any transport drop or replica death between consume and completion produces it.

3. Repro (standalone, no app code)

Requires only the two packages: pip install "agent-framework-core @ git+https://github.com/microsoft/agent-framework.git@main#subdirectory=python/packages/core" "agent-framework-ag-ui @ git+https://github.com/microsoft/agent-framework.git@main#subdirectory=python/packages/ag-ui" httpx

import asyncio, sys
from typing import Any

from agent_framework import (Agent, BaseChatClient, ChatMiddlewareLayer, ChatResponse,
                             ChatResponseUpdate, Content, FunctionInvocationLayer, Message, tool)
from agent_framework._types import ResponseStream
from agent_framework_ag_ui import AgentFrameworkAgent

EXECUTIONS: list[str] = []

@tool(approval_mode="always_require")
def deploy_change(target: str) -> str:
    """Deploy a change to the target environment."""
    EXECUTIONS.append(target)
    return f"deployed to {target}"

class ScriptedClient(FunctionInvocationLayer, ChatMiddlewareLayer, BaseChatClient):
    """Turn 1 calls the gated tool; turn 2 fails like a provider 500 (or answers, with --success)."""
    def __init__(self, *, fail_on_resume: bool = True, **kwargs: Any) -> None:
        super().__init__(**kwargs)
        self.model_calls = 0
        self.fail_on_resume = fail_on_resume

    def _turn_contents(self, turn: int) -> list[Content]:
        if turn == 1:
            return [Content.from_function_call(call_id="call-1", name="deploy_change",
                                               arguments={"target": "prod"})]
        if turn == 2 and self.fail_on_resume:
            raise RuntimeError("simulated provider 500 on the resume run")
        return [Content.from_text("All done.")]

    def _inner_get_response(self, *, messages, stream, options, **kwargs):
        self.model_calls += 1
        turn = self.model_calls
        if stream:
            async def _stream():
                yield ChatResponseUpdate(role="assistant", contents=self._turn_contents(turn))
            return ResponseStream(_stream(), finalizer=ChatResponse.from_updates)
        async def _respond() -> ChatResponse:
            return ChatResponse(messages=[Message(role="assistant", contents=self._turn_contents(turn))])
        return _respond()

async def main(first_post_succeeds: bool) -> None:
    client = ScriptedClient(fail_on_resume=not first_post_succeeds)
    agui = AgentFrameworkAgent(agent=Agent(client=client, name="repro", tools=[deploy_change]))
    user_msg = {"id": "m1", "role": "user", "content": "please deploy to prod"}

    events0 = [e async for e in agui.run({"threadId": "t1", "runId": "r0", "state": {},
                                          "messages": [user_msg]})]
    iid = next(o.interrupts[0].id for e in events0
               if (o := getattr(e, "outcome", None)) and getattr(o, "interrupts", None))
    print(f"POST #0: interrupt {iid} surfaced; executions={EXECUTIONS}")

    resume = {"threadId": "t1", "state": {}, "messages": [user_msg],
              "resume": [{"interruptId": iid, "status": "resolved", "payload": {"accepted": True}}]}

    try:
        events1 = [e async for e in agui.run({**resume, "runId": "r1"})]
        print(f"POST #1: completed; executions={EXECUTIONS}")
    except BaseException as exc:
        print(f"POST #1: stream died ({exc}); executions={EXECUTIONS}  <- consumed AND executed")

    events2 = [e async for e in agui.run({**resume, "runId": "r2"})]
    codes = [e.code for e in events2 if getattr(e, "code", None)]
    print(f"POST #2: codes={codes}; executions={EXECUTIONS}; model_calls={client.model_calls}")

asyncio.run(main(first_post_succeeds="--success" in sys.argv[1:]))

Observed on main @ e39a8a2e:

POST #0: interrupt call-1 surfaced; executions=[]
POST #1: stream died (simulated provider 500 on the resume run); executions=['prod']  <- consumed AND executed
POST #2: codes=['APPROVAL_RESUME_NOT_FOUND']; executions=['prod']; model_calls=2

The tool ran ("prod" was deployed), the model call after it failed, and the retry is told No pending approval interrupt found for resume interruptId 'call-1' — never re-attempted (no third model call), answer gone. With --success (POST #1 completes normally), the duplicate POST #2 gets the identical APPROVAL_RESUME_NOT_FOUND instead of an idempotent "already resolved" answer.

4. Expected behavior

• A resume retry after a failed run re-attempts the resume (and reports the real failure if it persists) instead of reporting a lost question.
• A duplicate of a successfully applied resume is answered idempotently — replay the recorded outcome, or at minimum a distinct code (e.g. APPROVAL_RESUME_ALREADY_RESOLVED) — and never re-executes the tool or clears live approval state.
• APPROVAL_RESUME_NOT_FOUND remains the answer only for interrupt ids that were genuinely never pending (replay protection intact).

Code Sample

Error Messages / Stack Traces

Package Versions

agent-framework-core: 1.13.0, agent-framework-ag-ui: 1.0.1

Python Version

No response

Additional Context

5. Suggested fix shape

The single-use guarantee should survive; what needs to change is when the spend becomes irrevocable. Either of these keeps single-claim semantics:

1. Settle on completion (two-phase). At ingress, claim the entry (mark claimed with the run id — still single-claim, a concurrent second resume is rejected) instead of deleting it. Commit the claim when the run reaches RUN_FINISHED; roll it back when the run errors or the stream dies, so the same resume can be re-presented. Since the approved tool executes at ingress, the claim record should carry the executed function_results (they are in hand at _agent_run.py:1333) so a rolled-back retry re-uses them rather than executing twice — which makes the retry exactly-once with respect to tool side effects, something no downstream wrapper can achieve.
2. Outcome record + idempotent replay. Keep the eager consume, but record interrupt_id -> {outcome, run_id, executed results} with a short TTL at the moment of consumption. On a resume for a non-pending id, consult the record: succeeded → replay/ALREADY_RESOLVED; failed → restore the entry (single-claim on the record) and re-attempt, reusing the recorded results; no record → today's NOT_FOUND.

Option 1 is cleaner (one state machine instead of registry + side table), but option 2 is less invasive to the current pipeline ordering.

6. Relationship to existing issues

• Python: [Feature]: Make the AG-UI Approval State store pluggable #7082 (Approval State store pluggability): the review discussion there already converged on "a resume is single-use authority" and asked about compare-and-set semantics. This report is the complementary defect: it exists even with a perfectly durable store — the repro above uses the default in-memory store in a single process. Durability decides where the entry lives; this issue is about when it is spent.
• Python: [Bug]: function_approval_response is dropped from the request under service-side storage, so an approval-paused run never resumes #7125 (approval response dropped under service-side storage, closed): a different resume-path defect; mentioned only to note it does not cover this.
• Downstream we currently ship an app-side idempotency shim (snapshot the entry at consume, restore on retry-after-failure, answer duplicates gracefully). It works, but it cannot be exactly-once — the tool has already executed at ingress by the time anything can fail, and only upstream holds the results at that moment — and it has to couple to private seams (_run_common._extract_resume_payload, _strict_resume_entries, the store attributes) to exist at all.

[github-actions]

👋 Hi @antsok — thanks for the report! We weren't able to reproduce this against the current codebase, and it looks like it may already be addressed in a more recent release.

• You reported using: agent-framework-core 1.13.0, agent-framework-ag-ui 1.0.1

Could you try upgrading to the latest release and let us know if the issue still reproduces? If it does, please reopen this issue or leave a comment with a fresh snippet and stack trace and we'll take another look. 🙏

[antsok]

Retried as asked — it still reproduces, and I believe the version premise of the triage doesn't hold here:

• agent-framework-core 1.13.0 / agent-framework-ag-ui 1.0.1 — the versions I reported — are the latest published releases on PyPI as of today; there is no more recent release to upgrade to.
• The report was verified against main @ e39a8a2e, which is still the tip of main right now, so "already addressed in the current codebase" can't be the case — nothing has changed since the verification.

Fresh run, clean venv installed from the PyPI wheels (pip install agent-framework-core==1.13.0 agent-framework-ag-ui==1.0.1 httpx), using the self-contained script from the issue body, unchanged:

POST #0: interrupt call-1 surfaced; executions=[]
POST #1: stream died (simulated provider 500 on the resume run); executions=['prod']  <- consumed AND executed
POST #2: codes=['APPROVAL_RESUME_NOT_FOUND']; executions=['prod']; model_calls=2

And the duplicate-after-success twin (--success):

POST #0: interrupt call-1 surfaced; executions=[]
POST #1: completed; executions=['prod']
POST #2: codes=['APPROVAL_RESUME_NOT_FOUND']; executions=['prod']; model_calls=2

In case it helps to pin down why an attempted repro might pass: the defect only shows on the second POST of the same resume. A single resume round-trip — gate a tool, answer the interrupt with resume: [{interruptId, status: "resolved", payload: {accepted: true}}], let the run complete — succeeds every time; the consume itself works. The failure needs either (a) a failure injected after ingress on the resume run (in the script, the chat client raises on the model call that follows the ingress-executed tool), then the identical resume re-POSTed, or (b) a completed resume run followed by re-POSTing the identical resume. Both retries answer APPROVAL_RESUME_NOT_FOUND (_agent_run.py:982/:1001/:1036 at e39a8a2e), indistinguishable from an interrupt id that never existed — while in case (a) the approved tool's side effect has already happened.

Happy to re-verify against any branch or pre-release you'd like.