Customer-Facing Model Cards for OpenAI-Compatible AI SDKs (August 2026)

[chawkian]

One production detail I would like more AI SDKs and gateways to expose is a customer-facing model card for every public route. Not a long marketing page, and not just the id field returned by /v1/models. I mean a small, dated contract that tells a developer what a route can do, how it is billed, which limits apply, and which source was checked.

This matters more for OpenAI-compatible Chinese model access because one client can hide very different upstream behavior. A DeepSeek route, a Kimi route, a GLM route, and a Qwen route may all accept chat-style requests, but they can differ in context length, output ceilings, cache accounting, tool support, rate limits, regional policy, and invoice math. If those fields are not visible until after traffic starts, the SDK has made routing feel simpler while making operations harder.

The dated price snapshot I checked today, 2026-08-19, from the live AIWave pricing endpoint includes these example rows:

• deepseek-v4-flash: $0.319/M input, $0.957/M output, $0.0318/M cache-hit input.
• deepseek-v4-pro: $0.957/M input, $2.871/M output, $0.0333/M cache-hit input.
• glm-5: $0.775/M input, $2.480/M output, $0.258065/M cache-hit input.
• kimi-k3: $2.250/M input, $11.250/M output, $0.200/M cache-hit input.
• qwen3.5-122b-a10b: $0.246575/M input and about $1.862/M output from its current completion ratio.

Those numbers should not become permanent constants in application code. They should become a versioned model-card snapshot with retrieved_at, source_url, currency, unit, public_model, upstream_model, and the pricing version when the gateway publishes one. The important habit is not memorizing today's table; it is making every table auditable later.

A useful card has four sections.

First, identity. The card should separate the public model string from the upstream provider and upstream model slug. If a gateway changes a backing route while keeping the same public name, the card history should make that visible.

Second, capabilities. The card should say whether the route supports streaming, JSON mode, tools, vision input, prefix caching, batch requests, and long context. A boolean is better than a paragraph because SDK tests can consume it. Unknown fields should be explicit null values rather than optimistic defaults.

Third, limits. Context tokens, maximum output tokens, RPM, TPM, concurrency class, timeout policy, and retry behavior all belong in the model card. An admission controller can reject impossible requests before they spend latency, and a support team can explain a rejection without reading private prompts.

Fourth, billing. A single blended price_per_token is not enough once cache rows, completion multipliers, provider schedules, and gateway groups exist. The card should preserve input, cached input, output, and any route class that changes the bill. Usage events can then store the card version instead of copying every price field onto every request.

Here is the minimal shape I would want an SDK to expose:

{
  "public_model": "deepseek-v4-flash",
  "provider": "AIWave",
  "upstream_model": "deepseek-v4-flash",
  "retrieved_at": "2026-08-19T14:00:00Z",
  "source_url": "https://aiwave.live/api/pricing",
  "capabilities": {
    "streaming": true,
    "tools": true,
    "json": true,
    "vision": false
  },
  "pricing": {
    "unit": "1M tokens",
    "input_usd": 0.319,
    "cached_input_usd": 0.0318,
    "output_usd": 0.957
  }
}

I would keep this endpoint separate from /v1/models. The model list stays compatible with existing OpenAI-style clients. A richer /v1/model-cards or SDK helper can carry operational metadata without breaking simple integrations.

The card also helps with governance. Public metadata should never include customer prompts, API keys, private fallback priorities, account discounts, or payment identifiers. It can still expose enough for developers and buyers to reason about tradeoffs: source date, policy region, retention class, supported features, and reproducible billing rows.

The discussion question: should an OpenAI-compatible SDK treat model cards as first-class API objects, or is this better left to docs and dashboards? I lean toward first-class objects because tests, admission control, support tooling, and billing reconciliation all need the same fields. But I am interested in where teams draw the boundary.

If you have built something similar, which fields turned out to be essential? Which fields created noise? And would you expect a Python SDK to cache model cards locally, fetch them live, or support both with an explicit snapshot version?