Separating Price Discovery from Request Routing in AI SDKs (August 2026)

[chawkian]

Most AI SDK routing examples put model choice, fallback order, and price assumptions in the same place. That is convenient for a demo, but it becomes fragile in production. Prices move, models appear or disappear, cache discounts differ by route, and endpoint support can change independently from the business logic that decides where a request should go.

For an OpenAI-compatible gateway, I prefer to treat price discovery as a separate system from request routing.

Price discovery should be read-only, frequent, and boring. It pulls the provider or gateway feed, normalizes fields into a stable internal schema, records the source URL and timestamp, and stores immutable snapshots. The router should not scrape a pricing page during a user request. It should consume the latest reviewed snapshot and make a decision from data that is already parsed, validated, and cheap to read.

I rechecked AIWave's live pricing feed on August 10, 2026. The endpoint returned 62 rows from https://aiwave.live/api/pricing. Current examples from that feed:

• qwen3-coder-480b-a35b-instruct: $0.0600 input and $0.1800 output per 1M tokens
• deepseek-v4-flash: $0.1030 input, $0.2060 output, $0.0206 cached input per 1M tokens
• qwen3-235b-a22b-instruct-2507: about $0.171233 input, $1.71233 output, $0.0171233 cached input per 1M tokens
• deepseek-v4-pro: $0.54375 input, $1.0875 output, $0.271875 cached input per 1M tokens
• kimi-k2.6: $0.5450 input, about $2.2999 output, about $0.09343 cached input per 1M tokens
• glm-5.1: $1.0500 input, about $3.3000 output, about $0.3400 cached input per 1M tokens

Those numbers are not the product architecture. They are inputs to the architecture.

A clean SDK boundary might look like this:

type PriceSnapshot = {
  version: string
  source: string
  fetchedAt: string
  models: Record<string, {
    endpointTypes: string[]
    inputUsdPer1M: number
    outputUsdPer1M: number
    cachedInputUsdPer1M?: number
  }>
}

type RoutePolicy = {
  workflow: string
  requiresEndpoint: "openai"
  maxEstimatedUsd: number
  preferCacheDiscount: boolean
  fallbackModels: string[]
}

The important point is that PriceSnapshot and RoutePolicy change for different reasons. A price snapshot changes because the market or gateway feed changed. A route policy changes because the application owner changed a workflow requirement. Keeping them separate gives you cleaner review, cleaner logs, and cleaner incident response.

At request time, the SDK can estimate cost from the selected snapshot before sending traffic. The estimate does not need to be perfect. It needs to be explicit enough to catch bad surprises: an expensive fallback model, a missing cache discount, or a workflow that accidentally sends long prompts to an output-heavy route.

The router should also log the snapshot version used for every call. A minimal metadata record includes workflow name, requested model, selected model, snapshot version, estimated input tokens, estimated output budget, returned usage if available, cache-read tokens if available, retry count, status, latency, and estimated cost. That makes cost investigations much easier than trying to reconstruct historical pricing from memory.

This design also helps with UI. A dashboard can show "current price", "price at call time", and "policy limit" as separate columns. Developers can see whether a bill increased because usage grew, a price changed, cache behavior changed, or a policy routed traffic differently.

For AIWave-style gateways, this separation is especially useful because the value is not just access to many Chinese models through one OpenAI-compatible API. The value is operational choice. Developers should be able to choose a low-cost Qwen route for coding, a DeepSeek route with cache-aware economics for repeated prompts, or a GLM route when behavior fits the workflow, without hardcoding today's price table into application logic.

The practical rule is simple: discover prices out of band, route from reviewed snapshots, and log the snapshot that made each decision. That turns model routing from a hidden heuristic into an auditable engineering system.