Preflight Fit Probes for OpenAI-Compatible SDK Routes (September 2026)

[chawkian]

One SDK pattern I would like to see become more common is a preflight fit probe before the first expensive model request.

Most OpenAI-compatible clients make it easy to swap base_url, model name, and API key. That is useful, but it also hides a practical question: should this workload run through this route at all today? A trial can fail for reasons that have nothing to do with JSON parsing. The account group may be wrong, the route may not expose the endpoint family the application expects, cached input may not be priced the way the team assumed, or a long-context job may exceed the output budget before anyone notices.

I checked the current AIWave pricing endpoint for this discussion on Sep 1, 2026. The live /api/pricing response returned success=true, 63 route records, pricing version a42d372ccf0b5dd13ecf71203521f9d2, auto_groups=["default"], and group ratios of default=3 and vip=1. A few parsed gateway examples before account-group math were:

Model	Input / 1M	Cache-hit input / 1M	Output / 1M
deepseek-v4-flash	$0.638	$0.020288	$1.914
deepseek-v4-pro	$1.914	$0.063736	$5.742
glm-5.1	$2.10	about $0.680001	$6.60
kimi-k3	$4.50	$0.90	$22.50

Those rows are not a recommendation by themselves. They are inputs to a decision that an SDK can help make visible. Before a production runner sends the real prompt, it could ask the gateway for a compact capability and pricing snapshot, then compare that snapshot against the workload contract.

For example, an SDK-level preflight result might look like this:

{
  "requested_model": "deepseek-v4-pro",
  "endpoint_family": "chat_completions",
  "pricing_version": "a42d372ccf0b5dd13ecf71203521f9d2",
  "effective_group": "vip",
  "supports_openai_endpoint": true,
  "max_input_tokens_planned": 180000,
  "max_output_tokens_planned": 12000,
  "cache_hit_expected": true,
  "estimated_stop_reason": null
}

The important behavior is not the exact JSON shape. It is that the SDK can stop early with a typed reason. Examples:

1. endpoint_family_mismatch if the application wants Responses-style behavior but the selected route is only being tested through Chat Completions.
2. billing_group_unconfirmed if the request would be estimated with a default multiplier while the customer expects VIP accounting.
3. cache_assumption_missing if the workload plan depends on repeated-prefix economics but the call has no cache key or cache policy.
4. output_budget_exceeded if the planned maximum output would make the trial unhelpful for a small evaluation balance.
5. route_identity_unpinned if a model alias could move without the application storing served model and pricing version.

This would keep a clean boundary between SDK ergonomics and gateway policy. The gateway still owns the live model table, account groups, and rate-card version. The SDK owns the developer-facing contract: "given this workload, this route, and this account state, is the first real request likely to produce interpretable evidence?"

I would probably expose this as an optional method rather than adding latency to every normal call:

probe = client.aiwave.routes.preflight(
    model="deepseek-v4-pro",
    endpoint_family="chat_completions",
    max_input_tokens=180_000,
    max_output_tokens=12_000,
    requires_cache_accounting=True,
)

if not probe.ok:
    raise RuntimeError(probe.stop_reason)

For trial users, this turns "the first request failed" into a more useful answer. For production users, it gives CI a place to reject route drift before the workload runs. For SDK maintainers, it may reduce support tickets where the response succeeded but the evidence was not usable.

Discussion question: should route-fit checks live inside SDKs, or should they remain a dashboard and documentation concern? I lean toward SDK-level opt-in, because the application already knows the planned workload shape before the gateway sees the real prompt.