Cache Prefix Version Receipts for OpenAI-Compatible SDKs (September 2026)

[chawkian]

Pricing/status note checked during the September 24, 2026 run: https://aiwave.live/api/v1/pricing returned HTTP 200 with 56 public model rows, currency USD, unit per_1m_text_tokens, checked=2026-09-10, updated_at=2026-09-18, and pricing version 83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5. The live route-pricing endpoint https://aiwave.live/api/pricing returned HTTP 200 with success=true, 79 route rows, pricing version a42d372ccf0b5dd13ecf71203521f9d2, and group_ratio.default=1, group_ratio.vip=0.9. The public status metrics endpoint was generated at 2026-09-24T04:35:01+00:00.

Long-context SDK examples usually talk about prompt caching as a cost optimization. That is true, but it is only half of the engineering problem. The other half is proving which prefix was cached, which version of that prefix was intended, and when a cache hit should stop being trusted.

I would like to see more OpenAI-compatible SDKs emit a small cache-prefix receipt beside each request. It does not need prompt text. It should describe the deterministic parts of the request that matter for budget review:

{
  "model": "kimi-k3",
  "endpoint": "openai.chat.completions",
  "prefix_id": "support-triage-system-v4",
  "prefix_hash": "sha256:...",
  "prefix_version": "2026-09-24.1",
  "cache_policy": "reuse_until_hash_or_policy_change",
  "input_tokens": 180000,
  "cached_input_tokens": 150000,
  "output_tokens": 1200,
  "pricing_snapshot_version": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5",
  "route_pricing_version": "a42d372ccf0b5dd13ecf71203521f9d2",
  "billing_group": "default",
  "redacted": true
}

The receipt is useful because prefix caching is fragile in a way that ordinary token counters do not show. A developer can change one sentence in a system prompt, reorder a policy block, switch a retrieval template, or add a hidden tool instruction. The request may still succeed, but yesterday's cache assumptions are no longer valid. If the SDK only records total input tokens, the cost review has to guess why the bill moved.

A prefix receipt gives the review three concrete checks.

First, did we use the expected prefix? The SDK can compare prefix_id, prefix_hash, and prefix_version against a pinned manifest before sending a production request.

Second, did the observed cache behavior make sense? If input_tokens is large and cached_input_tokens suddenly falls to zero, the CI report can flag a cache invalidation event instead of calling it random variance.

Third, can the cost estimate be reproduced later? Today's public AIWave snapshot lists kimi-k3 at $4.50 input, $0.90 cache-hit input, and $22.50 output per 1M text-token units, with row effective date 2026-08-27. The same checked snapshot lists deepseek-v4-flash at $0.638 input, $0.0202884 cache-hit input, and $1.914 output. Those are dated facts, not permanent promises, so the receipt should carry both the price source and the version used for the run.

I would also keep the cache receipt independent from reliability metrics. The public status snapshot can be current while still leaving end-to-end success rate or historical uptime unpublished when the denominator is incomplete. That is fine. Cache evidence answers a narrower question: did this request reuse the stable part of the prompt under the intended model, route, and billing context?

For SDK authors, the implementation can stay small:

def cache_prefix_receipt(prefix: str, *, prefix_id: str, version: str) -> dict:
    return {
        "prefix_id": prefix_id,
        "prefix_hash": sha256(prefix.encode("utf-8")).hexdigest(),
        "prefix_version": version,
        "redacted": True,
    }

The important rule is that the hash is computed before the request, while token and pricing fields are attached after the gateway returns usage and receipt metadata. That keeps prompt content private while giving support, finance, and release engineering enough evidence to explain cache misses.

My preference is to make this a first-class SDK fixture rather than an observability afterthought. Long-context workloads are exactly where small invisible changes become expensive. A redacted prefix-version receipt turns that invisible change into something a test can catch.

Discussion question: should OpenAI-compatible SDKs standardize cache-prefix receipt fields, or is this better handled as a gateway-specific extension?