Cache-Hit Regression Fixtures for Long-Context SDK Runs (September 2026)

[chawkian]

Long-context applications often treat cache behavior as an infrastructure concern. For an OpenAI-compatible SDK, I think it also deserves a test fixture. If a team depends on repeated system prompts, large retrieved context, or multi-turn planning traces, a cache regression can change the bill and the incident story even when every request still returns 200.

I checked the live pricing sources for this discussion on September 7, 2026. AIWave's public /api/pricing endpoint returned success=true, 63 route records, pricing version a42d372ccf0b5dd13ecf71203521f9d2, auto_groups=["default"], and group ratios of default=3 and vip=1. Selected route rows from that snapshot were:

Model	model_ratio	completion_ratio	cache_ratio	Enabled groups
deepseek-v4-flash	0.319	3	0.0318	vip, svip, default
deepseek-v4-pro	0.957	3	0.0333	default, vip, svip
glm-5.1	1.05	3.142857	0.32381	svip, default, vip
kimi-k3	2.25	5	0.2	default, vip, svip
qwen3-max	0.7810988945590882	4	absent	default, vip, svip
qwen3.6-flash	0.13390266763870082	6	absent	vip, svip, default

I also rechecked direct provider pricing pages the same day. DeepSeek lists token prices per 1M tokens and separates cache-hit input, cache-miss input, and output rows: deepseek-v4-flash is listed at $0.007 cache-hit input, $0.22 cache-miss input, and $0.66 output off-peak, with peak rows at $0.014, $0.44, and $1.32. deepseek-v4-pro is listed at $0.022, $0.66, and $1.98 off-peak, with peak rows at $0.044, $1.32, and $3.96. Z.AI lists GLM text rows per 1M tokens; GLM-5.1 is listed at $1.4 input, $0.26 cached input, and $4.4 output. QwenCloud documents per-million-token billing with separate input and output rates, tiered input pricing on some models, Batch API at 50% of real-time token rates, model-specific cache discounts, and a rule that batch and cache discounts are not combined on the same request.

Those facts point to a useful SDK boundary: cache behavior should be observable without making the SDK the billing authority. The gateway remains the source of truth for final debit rows, effective group, route policy, and provider mapping. The SDK can still help developers prove that a long-context workload preserved the same cache assumptions across releases.

One fixture shape could be:

{
  "fixture": "long_context_cache_regression_2026_09_07",
  "source_url": "https://aiwave.live/api/pricing",
  "pricing_version": "a42d372ccf0b5dd13ecf71203521f9d2",
  "requested_model": "deepseek-v4-pro",
  "effective_group": "vip",
  "stable_prefix_hash": "sha256:...",
  "expected_cache_fields": ["cache_ratio"],
  "required_receipt_fields": [
    "resolved_model",
    "pricing_version",
    "input_tokens",
    "cached_input_tokens",
    "output_tokens"
  ]
}

The test should run in two phases. First, send a deterministic warm-up request with a stable prefix, low temperature, and a small output cap. Store only metadata: prefix hash, model, checked time, pricing version, route group, token counts, and whether cached-input fields appeared. Do not store prompts, completions, credentials, account emails, payment identifiers, or raw private documents.

Second, send a follow-up request with the same prefix and a small suffix change. The fixture should not assert an exact dollar amount unless the gateway exposes a final ledger row for that request. It should assert the fields that make later analysis possible: the same resolved model, same pricing version or an explicitly recorded version change, same effective group, visible cached-input accounting when the route promises it, and a typed warning when the selected route has no cache metadata.

The failure modes should be explicit. Fail closed when the requested model disappears, the effective group is not enabled for the route, the pricing version is unknown, or a route that previously exposed cache fields silently drops them. Warn when the provider's public page changed but the gateway snapshot has not yet changed, because that may be a review queue item rather than an SDK runtime failure.

In Python, the helper can stay narrow:

receipt = client.aiwave.cache_regression_probe(
    model="deepseek-v4-pro",
    stable_prefix=system_prompt,
    suffix="short diagnostic request",
)

receipt.assert_contract(
    pricing_version="a42d372ccf0b5dd13ecf71203521f9d2",
    effective_group="vip",
    required_cache_fields=["cached_input_tokens"],
)

This would make cache-hit regressions visible in CI, rollout checks, and support diagnostics. It also keeps public evidence clean: source URLs, checked dates, pricing versions, model IDs, and redacted receipts are enough. The SDK should help teams preserve the facts they need to explain a long-context run without turning private workload content into a test artifact.

Open question: should aiwave-sdk-python expose this as a typed cache regression helper, or publish it first as a reference fixture under examples?