Tool Result Budgets for OpenAI-Compatible Agent SDKs (August 2026)

[chawkian]

One SDK control I would like to see treated as a first-class request option is a tool result budget. Most agent cost controls focus on the initial prompt, the output cap, or the selected model. That misses a large source of drift: the text that comes back from tools and re-enters the model context.

An agent may start with a small user message, then call a search tool, a ticket lookup, a repository scanner, a database summarizer, or an MCP server. Each tool may return far more content than the user wrote. If the SDK only tracks max_tokens, the gateway may accept a request that becomes expensive after the first tool call. The caller needs a way to say: this run can use tools, but tool results may only add a bounded amount of context.

For today's check, August 25, 2026, I pulled the live AIWave /api/pricing feed before writing. It returned success=true, pricing_version=5a90f2b86c08bd983a9a2e6d66c255f4eaef9c4bc934386d2b6ae84ef0ff1f1f, and group_ratio values of default=3 and vip=1. Example VIP-baseline rows from that feed:

• deepseek-v4-flash: $0.319/M input, $0.957/M output, $0.0318/M cached input.
• deepseek-v4-pro: $0.957/M input, $2.871/M output, $0.0333/M cached input.
• glm-5: $0.775/M input, about $2.480/M output, $0.258065/M cached input.
• kimi-k3: $2.250/M input, $11.250/M output, $0.200/M cached input.
• qwen3.5-122b-a10b: $0.246575/M input and about $1.863/M output.

I also rechecked the public AIWave pricing and predictable-pricing pages, DeepSeek pricing docs, QwenCloud pricing docs, Kimi K3 page, and Z.AI pricing docs. They all returned HTTP 200 during the run. Those pages matter because tool-heavy agents can involve output caps, cache-hit input, batch policy, thinking output, and provider-side tool fees. Prices should be treated as dated source inputs, not constants in SDK code.

The SDK shape I would start with is explicit:

{
  "tool_budget": {
    "max_tool_calls": 4,
    "max_result_tokens_total": 12000,
    "max_result_tokens_per_call": 4000,
    "count_result_tokens_as": "input",
    "truncate_strategy": "fail_closed",
    "pricing_source": "https://aiwave.live/api/pricing",
    "pricing_checked_at": "2026-08-25",
    "pricing_version": "5a90f2b86c08bd983a9a2e6d66c255f4eaef9c4bc934386d2b6ae84ef0ff1f1f"
  }
}

The important field is not only max_tool_calls. A single successful tool call can be worse than several small calls if it returns a long document, a verbose stack trace, or a wide JSON payload. max_result_tokens_total protects the whole run. max_result_tokens_per_call catches one oversized response early. truncate_strategy defines whether the SDK should trim results, summarize them, or fail before sending a second model call.

My preference is fail_closed for production workflows until the application has reviewed the data shape. Silent truncation can create subtle correctness bugs. A support agent may miss the part of a policy document that matters. A code agent may drop the file that explains a failing test. If a team does allow truncation, the SDK should include an observable marker such as tool_result_truncated=true, the original byte count, the retained token count, and the tool name.

The gateway should estimate the second model call before it happens. It already has the selected route, pricing version, cache assumption, and output cap. After a tool returns, the SDK can compute a revised estimate:

fresh_input = original_input + uncached_tool_result_tokens
cached_input = stable_prefix_tokens + reusable_tool_result_tokens
output_exposure = max_output_tokens + max_thinking_tokens

If the revised estimate crosses the caller's budget ceiling, the SDK should stop with a structured error. That is better than allowing a planner loop to keep turning tool output into model input.

This also gives support and finance better records. Each run should log tool_name, call_index, result_tokens, truncated, route, pricing_version, estimated_incremental_usd, and decision_reason. Those fields explain why an agent run was accepted, blocked, or shortened.

Discussion question: should tool result budgets live in the Python SDK request object, in the gateway policy, or both? My bias is both. The SDK can make tool output visible to developers before the next model call. The gateway can enforce the freshest rate card and account-level limits.