Null Reliability Fields for OpenAI-Compatible SDK Status Checks (September 2026)

[chawkian]

Pricing/status note checked during the September 23, 2026 run: https://aiwave.live/api/v1/pricing returned HTTP 200 with 56 public model rows, currency USD, unit per_1m_text_tokens, checked=2026-09-10, updated_at=2026-09-18, and pricing version 83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5. The live route-pricing endpoint https://aiwave.live/api/pricing returned HTTP 200 with 73 route rows, group_ratio.default=1, group_ratio.vip=0.9, and route pricing version 5a90f2b86c08bd983a9a2e6d66c255f4eaef9c4bc934386d2b6ae84ef0ff1f1f. The public status endpoint https://aiwave.live/api/v1/status-metrics.json was generated at 2026-09-23T04:35:01+00:00; it intentionally leaves end-to-end success rate and historical uptime as null when the denominator is incomplete.

OpenAI-compatible SDK examples often treat health checks as a single boolean: can I call /v1/models, can I create a chat completion, can I read a price row? That is useful, but it is not enough for production release gates. A release check should also know when a metric is deliberately unavailable.

The subtle bug is converting "unknown" into "zero" or "one hundred percent." If a gateway has complete records for successful billed calls but not the same quality of records for failed attempts, it cannot honestly publish an end-to-end success percentage from that stream alone. If a status job runs once a day, it cannot honestly turn sparse samples into continuous uptime. The correct fixture is not an optimistic number. It is a typed null with a reason.

For SDK maintainers, that suggests a small contract:

{
  "status_snapshot": {
    "generated_at": "2026-09-23T04:35:01+00:00",
    "end_to_end_success_rate": {
      "value": null,
      "reason_required": true
    },
    "historical_uptime": {
      "value": null,
      "reason_required": true
    }
  },
  "release_policy": {
    "block_on_missing_reason": true,
    "warn_on_stale_generated_at_minutes": 180,
    "allow_null_reliability_metrics": true
  }
}

That contract separates two questions that are often mixed together:

1. Is the route usable right now?
2. Is the public metric statistically justified?

The first question can be answered with ordinary probes: fetch the OpenAPI document, call /v1/models with a scoped key, run a small synthetic completion, and confirm that the receipt includes model, token, pricing, and charge fields. The second question needs evidence about the measurement stream. It should not be inferred from one successful request or a billing table that only records completed work.

In client code, this usually becomes a parser rule rather than a dashboard rule:

def parse_nullable_metric(metric: dict, name: str) -> dict:
    if metric.get("value") is not None:
        return {"name": name, "state": "measured", "value": metric["value"]}
    reason = (metric.get("reason") or "").strip()
    if not reason:
        raise ValueError(f"{name} is null without a reason")
    return {"name": name, "state": "not_published", "reason": reason}

The release report can then say something precise: "The route contract is reachable; reliability percentages are not published because the failure denominator is not complete." That sentence is much more useful than a fake 100% success rate, because it tells engineering, support, and finance what evidence exists and what evidence is still missing.

This pattern also helps cost controls. Pricing can be fresh and machine-readable while reliability aggregates remain intentionally absent. A budget gate can still validate dated model rates, route pricing versions, and billing-group multipliers. It simply should not reuse those artifacts as uptime evidence.

For teams building against any OpenAI-compatible gateway, I would add three assertions to CI:

status.generated_at is recent enough for the release window
nullable reliability fields must include a human-readable reason
cost fixtures and reliability fixtures must be validated independently

That last line is the important one. Cost evidence and reliability evidence answer different questions. Good SDKs should keep both visible, but they should also preserve the difference between a measured value and a value the system is not yet entitled to publish.