Pre-Dispatch Cost Estimates for Multi-Model AI SDKs (August 2026)

[chawkian]

One small SDK feature I would like to see become normal is a pre-dispatch cost estimate for every routed request. Not a billing dashboard, and not a finance report after the invoice arrives. I mean a compact estimate generated before the HTTP call leaves the application: selected model, expected token mix, pricing snapshot, cache assumption, and fallback budget.

This is especially useful for OpenAI-compatible gateways that route across Chinese models. The application code may send the same chat.completions.create() call every time, while the route table is deciding among DeepSeek, Qwen, Kimi, GLM, ERNIE, and MiniMax rows with different cache behavior and output multipliers. If the SDK does not expose the estimate up front, teams often discover cost drift only after usage exports are reconciled.

I rechecked AIWave's live pricing feed on August 15, 2026 (https://aiwave.live/api/pricing). Current OpenAI-compatible rows included:

Model	Input / 1M	Cached input / 1M	Output / 1M
qwen3-coder-480b-a35b-instruct	$0.0600	n/a	$0.1800
deepseek-v4-flash	$0.1030	$0.0206	$0.2060
qwen3-235b-a22b-instruct-2507	$0.171233	$0.0171233	$1.71233
minimax-m2.5	$0.2500	n/a	$1.0000
kimi-k2.6	$0.5450	about $0.09343	$2.2999
glm-5.1	$1.0500	about $0.3400	about $3.3000
ernie-5.0	about $1.232877	about $0.123288	about $4.65756

Those rows are enough to make a route estimate meaningful before dispatch. For example, an SDK could attach a local object like this to logs, traces, or an optional response metadata field:

{
  "route": "coding-agent-edit",
  "selected_model": "deepseek-v4-flash",
  "pricing_checked_at": "2026-08-15T13:00:00Z",
  "price_source": "https://aiwave.live/api/pricing",
  "estimated_prompt_tokens": 42000,
  "estimated_cached_tokens": 31000,
  "estimated_output_tokens": 1800,
  "estimated_cost_usd": 0.00157,
  "fallback_budget_usd": 0.00400,
  "fallback_candidates": [
    "qwen3-coder-480b-a35b-instruct",
    "glm-5.1"
  ]
}

The useful part is not the exact decimal precision. It is the contract: before the SDK calls a model, the application can decide whether the request fits its route budget, whether cache assumptions are reasonable, and whether fallback expansion should be allowed. That decision is much easier before dispatch than after a streaming response has already completed.

I would keep this separate from billing reconciliation. A reconciliation harness should compare estimates against provider usage rows later. A pre-dispatch estimate has a narrower job: protect the hot path from stale route tables, unexpected prompt growth, cache-key misses, and fallback cost surprises.

There are a few design questions I keep coming back to:

1. Should SDKs return these estimates only in debug mode, or should they be available as normal structured metadata?
2. Should the estimate be generated before retries and fallbacks, then updated after the final model is selected?
3. How should SDKs represent unknown cache state: pessimistic cache miss, optimistic cache hit, or a range?
4. Should applications be able to reject a routed request when the pre-dispatch estimate exceeds a per-route budget?

My current preference is a small, opt-in metadata object with a stable schema and a dated pricing source. That gives developers enough evidence to debug route decisions without turning every response into an accounting document.

How are you handling this in your own SDKs or gateways? Do you prefer pre-dispatch estimates in response metadata, structured logs, OpenTelemetry spans, or a separate cost-control callback?