feat: credential-aware upstream selection on proxy-on-miss (refuse mismatches, route GitHub Models)

[contextablemark]

Problem

Two related failure modes on a fixture miss in --proxy-only / --record mode, both rooted in aimock picking the proxy upstream by request surface while the caller's credential may belong to a different provider:

1. Credential leak to the wrong vendor (PNI-108). aimock forwards the caller's credential verbatim to the configured upstream. When the credential clearly belongs to a different provider than the upstream — e.g. a GitHub Models token (github_pat_…) reaching api.openai.com because a fixture was missing and --proxy-only fell through — the real credential is relayed to the wrong vendor, which rejects it with invalid_api_key. aimock normalizes that to a 502: a confusing OpenAI auth error, and the credential has already left in transit.

2. Legitimate GitHub-Models fall-through 502s (PNI-110). GitHub Models is OpenAI-wire-compatible — clients point a stock OpenAI SDK at https://models.inference.ai.azure.com with a GitHub PAT as the bearer token — so aimock classifies the request as openai. On a single shared aimock instance, --provider-openai is one global URL (it can't be api.openai.com for real-OpenAI integrations and GitHub Models at the same time), so an unmatched/free-text GitHub-Models prompt that should legitimately fall through to a live LLM instead hits OpenAI and 502s. aimock must pick the upstream per request, not per process.

This is the failure mode we hit debugging the CopilotKit showcase: real browser traffic reached aimock without x-aimock-context, missed every context-scoped fixture, and got proxied to OpenAI carrying a GitHub token.

Fix — one mechanism, two outcomes

Credential-aware upstream selection. aimock inspects the outgoing credential and picks the upstream per request:

• Route (PNI-110). When the caller's bearer token is detected as a GitHub credential, the request is routed as openai, and a --provider-github upstream is wired, aimock reassigns the effective provider to a first-class github and forwards the PAT to GitHub Models. URL lookup, auth scheme, the compat guard, and egress metrics all realign off the effective key.
• Refuse (PNI-108). When the credential and the upstream host are confidently classified to different provider families and no matching upstream is configured (e.g. a GitHub token bound for api.openai.com), aimock refuses with a clear 502 proxy_refused — no upstream call, no leak.
• Fails OPEN on any uncertainty: unknown/dummy (sk-aimock-…) credentials, custom/self-hosted hosts, and signed/OAuth providers (Bedrock SigV4, Vertex OAuth) are never blocked, so legitimate/self-hosted setups are unaffected.
• Escape hatch: AIMOCK_ALLOW_CROSS_PROVIDER_PROXY=1.

Net: an unmatched GitHub-Models prompt falls through successfully to real GitHub Models when --provider-github is set, and is refused cleanly (never leaked to OpenAI) when it isn't. Route is the configured case; refuse is the general default of the same selection.

First-class github provider (PNI-110)

• --provider-github <url> CLI flag, AIMOCK_PROVIDER_GITHUB_KEY env var, bearer auth scheme, added to RecordProviderKey.
• Opt-in — inert unless --provider-github is wired, consistent with the existing providerKeys design.
• Credential-aware remap in proxyAndRecord via a new exported detectBearerCredentialProvider that reads the raw Authorization header, so no buildForwardHeaders reorder on the hot path.
• Fixture recording stays under the original surface key (openai) so replay — which re-classifies by surface — still matches; only egress (upstream/auth/guard/metrics) uses the effective github key.
• The refusal message now names --provider-github instead of the misleading "point --provider-openai at the correct host" (impossible on a shared instance).

Guard internals (PNI-108)

• Pure helpers in provider-auth.ts: detectCredentialProvider (key-prefix → family), classifyUpstreamHost (host → family), checkCredentialUpstreamCompat (the fail-open decision). GitHub Models hosts (models.inference.ai.azure.com, models.github.ai) classify as github, so the github→github route is allowed and only the mismatched host is refused.
• Enforced in recorder.ts proxyAndRecord after applyProviderAuth, protecting every provider handler that proxies through that path. No runtime dependencies added.

Observability

• aimock_proxy_refused_total{provider,reason}
• aimock_proxy_forwarded_total{provider,context_present} — context_present surfaces requests proxying with no X-AIMock-Context (i.e. every context-scoped fixture missed).

Both labels use the effective provider, so a github-routed forward is labeled provider="github".

Tests

src/__tests__/credential-guard.test.ts — 38 tests: credential detection (incl. the new detectBearerCredentialProvider header helper), host classification, the compat decision (incl. the exact GitHub-token → api.openai.com case), fail-open branches, non-bearer schemes, the escape hatch, the --provider-github remedy message, and integration tests proving a GitHub PAT routes to the github upstream (not openai) while a real sk- key still goes to openai. The refusal integration test makes no real network call — the guard short-circuits before any socket is opened.

Full suite: 4806 passed / 3 skipped; prettier + eslint clean; tsdown build (typecheck) green.

Deployment

Inert until the shared showcase aimock instance is started with --provider-github https://models.inference.ai.azure.com. That single flag is what turns the unmatched-GitHub-Models 502s into successful live fall-through.

Related

• Linear: PNI-108 (the refuse half) + PNI-110 (the route half) — folded into one PR since PNI-108 had not merged and the two are two halves of one mechanism.
• PNI-109 removes the miss for curated demo prompts via x-aimock-context injection; this PR covers the residual unmatched/free-text GitHub-Models prompts that legitimately fall through to a live LLM.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@340

commit: f675b7f

[linear]

PNI-108

[contextablemark]

There's a workaround in the MS Harness Agent that avoids the need for the Github Models endpoint. I'll use that instead. Closing this for now.