Python: [Feature]: FIDES integration: improvement requests from a production deployment

[antsok]

Description

Summary

We integrated agent_framework.security into a multi-surface agent app, shipped it, hit a production incident within hours, and hardened it. The taint model itself did what it promises — the integrity leg caught real shapes, and the classification approach is sound. Everything below is about integration surface: places where an integrator has to reach around the API, or where behavior differs from what the shape of the API implies.

Each item states the workaround it currently costs an integrator. Every one of them exists only until the corresponding upstream gap closes.

Context that makes several of these bite harder than they might in a sample: this app runs FIDES across four surfaces (an AG-UI web host, DevUI, an OpenAI-compatible API + Telegram, and a terminal console), shares one Agent per model across many concurrent conversations, and runs FIDES inside its background sub-agents as well as the top-level agent — so the advisor and its children share one security boundary.

---

A. Bugs

A1 — Tracked state is per middleware instance, so one Agent serving many conversations cross-contaminates

LabelTrackingFunctionMiddleware._context_label (security.py:771) and PolicyEnforcementFunctionMiddleware._pending_policy_approvals (security.py:1704) are instance attributes, not session state.

Any host that shares one Agent across conversations therefore shares the label. That is not an exotic setup — it is what MAF's own AG-UI hosting encourages, and what we do deliberately: one agent per catalog model, reused by every conversation on it, with all per-conversation state in AgentSession (as the rest of the harness does). With FIDES wired in, one user's taint leaks into every other conversation on that model for the process lifetime.

The second-order effect is worse than over-blocking. _matches_pending_approval (security.py:1824) re-derives its binding's context-label key from that same shared, drifting label at resume time, so unrelated concurrent traffic between a violation and its approval can make a legitimate approval fail to bind.

Ask: key tracked state by session (AgentSession state, like the rest of the harness), or expose a documented scoping hook.

Workaround: a wrapper middleware that keeps a per-conversation (tracker, enforcer) pair, LRU-bounded, plus a process-global tracker registry so an agent and its sub-agents resolve to one label.

Note: we key on our own conversation id rather than session.session_id, because a server-side-stateful provider (Azure Foundry / OpenAI Responses) reassigns session_id to the provider's conversation id mid-run. If upstream adopts session-scoping, that reassignment needs thinking about too — it would split a conversation's label halfway through a turn.

A2 — A blocked call ends the turn with empty output

_block_policy_violation (security.py:1926) sets the refusal on context.result and raises MiddlewareTermination. _tools._process_function_requests turns any termination into action="return" (_tools.py:2437), which ends the tool-invocation loop — so the model never gets a turn to react to the refusal.

Measured on a real run (three-turn script: call an unclassified tool, call a gated sink, then answer): two model turns instead of three, and response.text == "".

The refusal is delivered as a proper function_result, so nothing is orphaned and no provider rejects the payload. The problem is purely the termination: on a surface with no approval UI the user gets silence — an empty Telegram reply, an empty API response — with no indication that a policy stopped anything. In our case it also propagates through delegation: a blocked write inside a sub-agent ends that child's run empty, so the parent receives an empty report from a task that looks like it succeeded.

Ask: do not terminate the loop for a block. Return the refusal as an ordinary function result — upstream already constructs one — so the loop continues and the model can explain itself. If terminating is wanted for some callers, make it an explicit flag.

Workaround: swallow the termination for a block (never for an approval request, which must terminate so the card can surface) and hand the model the refusal plus explicit guidance telling it not to retry and to report the refusal honestly.

A3 — The turn field in every audit record is always -1

Both violation records read context.metadata.get("turn_number", -1) (security.py:2021 and :2044), and nothing in agent_framework ever sets turn_number — verified across the package: two reads, zero writes. So the audit log, which is the artifact you would use to review policy decisions after the fact, cannot say which turn a violation occurred on.

Confirmed in our deployed logs: every entry carries 'turn': -1.

Ask: populate it, or drop the field rather than emit a constant that reads like real data.

---

B. Extension points an integrator has to reach around

B1 — No way to declare labels for tools you did not construct

The documented idiom is @tool(additional_properties={...}) (FIDES announcement). It works only for tools the integrator authors.

In a harness-based app, most tools are not ours. file_access_*, file_memory_*, todos_*, mode_* and the skills tools are constructed inside create_harness_agent; microsoft_docs_* are expanded by the MCP client. That is 27+ tools with no place to pass additional_properties — and they include every privileged sink the policy is supposed to protect. Undeclared, they inherit the untrusted default and taint the conversation on the first todos_add.

So every integrator with a non-trivial tool surface has to write a middleware that mutates function objects at call time, before the tracker reads them. That is a lot of ceremony for "these tools are trusted", and it silently depends on middleware ordering.

Ask: a first-class mapping — e.g. SecureAgentConfig(tool_labels={"file_access_write": {...}}), or a label_tools(agent, mapping) helper — so declarations for third-party/harness tools live in configuration rather than in a hand-rolled middleware.

Workaround: a declaration middleware running outermost in the FIDES stack, which setdefaults the same keys the decorator would have set (an existing declaration always wins).

B2 — An approval-rule callback cannot tell a policy violation from an ordinary approval

ToolApprovalRuleCallback is Callable[[function_call], bool], and _function_call_from_request (_harness/_tool_approval.py:293) hands it only the reconstructed function_call — never the request's additional_properties (policy_violation, violation_type, …).

Consequence: any auto-approval rule that matches by tool name or arguments silently approves a FIDES policy violation as though it were the tool's ordinary approval reason. That includes rules written long before FIDES existed, for entirely unrelated purposes — ours auto-approve a file write to a path this conversation created (a low-noise UX rule).

Reverse-proofed: our real collect rule returns True for exactly the request shape a policy violation produces. This is the defect that turned one of our incidents fatal — the violation was auto-approved, the tool was replayed, and the run died on the provider.

Ask: pass the approval request (or at least its additional_properties) to the callback, or expose a public marker on the reconstructed call so a rule can opt out.

Workaround: subclass the enforcer to tag the reconstructed call, and wrap every rule with a guard that refuses when the tag is present.

B3 — A standing ToolApprovalRule bypasses every callback, including B2's fix

_matches_rule (_harness/_tool_approval.py:307) is a private module function consulted before any auto-approval callback runs. So a standing "always approve this tool" rule — created when a user clicked "always approve" for a tool's ordinary reason — pre-approves every later policy violation on that tool.

This one is not fixable from application code at all: there is no hook between the standing rule and the decision.

Ask: never match a standing rule against a request carrying policy_violation, or give ToolApprovalRule a scope/predicate so an integrator can express "ordinary approvals only".

Workaround (partial): on our terminal surface we withhold the "always approve" choices entirely for a policy violation, so no such rule can be created from that prompt. On the web surface, where "auto-approve similar" is part of the UI, the gap is still open.

B4 — The fix for B2 requires overriding a private method

Tagging the request means overriding PolicyEnforcementFunctionMiddleware._build_function_call_content (security.py:1725) — the single place the approval request's function_call is reconstructed.

Ask: a public extension point, or make it unnecessary by fixing B2.

B5 — allow_untrusted_tools can only express an allowlist

Our sink set is naturally a denylist: every harness tool keeps working in a tainted context except the file/memory mutations. An allowlist would have to enumerate every harness tool and would break the first time an unlisted one appeared — including on a version bump that adds one.

We get there by exploiting an undocumented two-touchpoint contract: the enforcer checks allow_untrusted_tools for truthiness at construction (allow or set()) and uses name in … per call. A set subclass that inverts __contains__ and forces __bool__ therefore behaves as a denylist. It works, but it is an accident of the implementation, so we pinned both touchpoints with contract tests that fail loudly on a pin bump.

Ask: support a denylist or a predicate first-class — deny_untrusted_tools=, or accept a callable for allow_untrusted_tools.

Workaround: a small allow-all-except adapter implementing the container protocol the enforcer probes.

---

C. Design and documentation

C1 — SecureAgentConfig is all-or-nothing

The sanctioned wiring injects the quarantine tools and a large variable-reference instruction block into the agent. A deployment that wants labels + policy but not quarantine (auto_hide_untrusted=False) cannot use it: for us, hiding tool results behind [var_…] placeholders would break the product, since the agent's entire job is reading, quoting and citing documentation.

So we hand-wire the two middlewares — which is what pushes us onto the private surface in B4/B5 in the first place.

Ask: let SecureAgentConfig compose without quarantine, or document hand-wiring as a supported path with the extension points it needs.

C2 — SecureAgentConfig keeps the quarantine client in a process-global, last-writer-wins slot

Upstream's own docstring documents this: multiple SecureAgentConfig instances in one process share whichever quarantine_chat_client was constructed last, and "if you need distinct quarantine clients per agent, run them in separate processes."

For a multi-tenant host that is a hard architectural constraint stated as a footnote. It is the same class of problem as A1 — per-process state where per-agent or per-session was needed.

Ask: per-instance resolution.

C3 — A read-only MCP tool gets no confidentiality cap

_map_mcp_annotations_to_labels (security.py:3018) maps readOnlyHint=True to accepts_untrusted=True and no max_allowed_confidentiality, documented as "pure data source, safe to call even when the context is tainted — it cannot exfiltrate".

The first half is right; the second is only right for the result. A read tool's arguments are egress: a search query can carry private content out of the app. That is precisely the exfiltration path we cap — our Microsoft Learn calls are read-only and still get an explicit confidentiality cap for that reason.

Ask: reconsider the default for tools whose arguments leave the process, or document that a query-carrying read needs its cap set explicitly rather than inheriting "no cap".

Code Sample

Language/SDK

Python