Claude Code: 400 context-length error after live model switch

[numikel]

What happened?

When switching Claude Code models live (tray slot submenu or settings), the next request sometimes fails with:

API Error: 400 This endpoint's maximum context length is 131072 tokens. However, you requested about 135293 tokens (36915 of text input, 34378 of tool input, 64000 in the output). Please reduce the length of either one, or use the context-compression plugin to compress your prompt automatically.

The failure is intermittent — it depends on how large the in-flight Claude Code session already is and which model the slot was switched to.

Expected

Live model swap should either keep working or surface a clear, actionable warning before/at swap time (e.g. “new model has a smaller context window than this session”).

Steps to reproduce

1. Configure a Messages endpoint (Claude Code) with slot models that have different context windows (e.g. switch from a ~200k model to one capped at 131072).
2. Launch Claude Code via the tray and work until the session accumulates substantial history (tool calls, file reads, long thread).
3. Without restarting Claude Code, switch one or more slots to the smaller-context model from the tray.
4. Continue the session — observe intermittent 400 with the context-length message above.

(Exact repro varies with provider catalog and session size; the error appears when input + tool_input + max_output exceeds the new model’s limit.)

App version

Observed on current main / v0.4.x Claude Code support (live slot swap via proxy-cc/* labels).

CLI agent

Claude Code (claude), launched through the proxy’s Run Claude Code flow.

Analysis (current behavior)

Today the proxy handles Claude Code like this:

• Launch: sets stable proxy-cc/<slot> labels in env; maps to catalog ids at request time (resolve_messages_model in proxy-core/src/proxy.rs).
• Live swap: only rewrites the outgoing model field — request body (including max_tokens) is otherwise pass-through.
• Token limits from /models: parsed into ModelInfo.max_prompt_tokens / max_output_tokens and used for Copilot (COPILOT_PROVIDER_MAX_*), but not applied to Claude Code requests or UI (token override panel is hidden for Messages API by design).

So after a swap, Claude Code keeps its existing conversation + tool context and may still request a large output budget (e.g. 64000). If the newly selected upstream model has a smaller combined limit (here 131072 total), the provider rejects the request even though the proxy successfully remapped the model id.

Ideas to explore

1. Proxy-side clamping (Messages API): when rewriting model, also adjust max_tokens (and possibly related fields) using the target model’s advertised limits from the catalog, ensuring prompt + max_output ≤ context_window where provider metadata allows inference.
2. Pre-swap guardrail: before accepting a tray/settings slot change, compare the new model’s limits against a heuristic or last-known session size; warn or block when likely to overflow.
3. Surface limits in CC UI: show per-slot model context/output caps in settings/tray so users can pick compatible models without guessing.
4. User guidance: document that switching to a smaller-context model mid-session may require /clear or restart; mention provider “context-compression” plugins where applicable.
5. Investigate upstream variance: some gateways report context_length vs max_output_tokens differently — confirm extract_token_limits mapping is correct for the providers users hit with CC.

Acceptance criteria (draft)

• Repro case documented with provider + two model ids
• Agreed strategy (clamp vs warn vs both)
• No silent 400s for the common “swap to smaller model, continue session” path — or an explicit in-app warning when that path is unsafe