Skip to content

providers: nvidia rejects unsupported prompt_cache_key sent by V2 (OpenAI-compatible runtime) #49240

Closed

Description

@anmarui74

Summary

OpenCode 2.0.3 sends the prompt_cache_key parameter to OpenAI-compatible providers by default. The NVIDIA NIM API (https://integrate.api.nvidia.com/v1) does not support it and rejects the request, so most NVIDIA models are unusable in V2.

Environment

  • opencode version: 2.0.3
  • OS: Linux 7.2.5-1-cachyos x86_64 (CachyOS)
  • Terminal: TERM=xterm-256color (Kitty), COLORTERM=truecolor
  • Shell: /usr/bin/zsh
  • Install/channel: release (CachyOS package opencode 2.0.3-1.1, channel=latest)
  • Active plugins: TUI plugins/sidebar-mimo/tui.tsx, opencode-voice-modified/tui.js, server plugins/nvidia-filter/index.ts, plus -opencode.sidebar.context in cli.json. (None related to the provider request path.)

Reproduction

  1. Configure the built-in nvidia provider (API key from https://integrate.api.nvidia.com/v1).
  2. Run any of the affected models headless:
    opencode run --standalone --model nvidia/meta/muse-glimmer-30b "di OK"
  3. Observe the error.

Expected Behavior

The request should succeed (or at least not include parameters the provider does not support). The same models return HTTP 200 when called directly against the NVIDIA API without prompt_cache_key.

Actual Behavior

Error: Validation: Unsupported parameter(s): `prompt_cache_key`

Affected built-in NVIDIA models (all HTTP 200 when called directly):
meta/muse-glimmer-30b, nvidia/nemotron-3-super-120b-a12b, nvidia/nemotron-3-ultra-550b-a55b, nvidia/nemotron-3.5-lightning-30b-a3b, nvidia/nemotron-3-nano-omni-30b-a3b-reasoning.

Models that still work (no prompt_cache_key sent): openai/gpt-oss-20b, meta/llama-3.2-11b-vision-instruct.

Additional Context

The bundled OpenAI-compatible chat runtime appears to add the parameter unconditionally unless cache === "none":

// request body
prompt_cache_key: e.cache === "none" ? void 0 : n.promptCacheKey ?? e.promptCacheKey
// and
Tb = (e) => { if (e.cache === "none" || e.promptCacheKey === void 0) return; ... }

There is a provider predicate that already excludes NVIDIA (g = n === "nvidia" || baseURL.includes("integrate.api.nvidia.com")) for some caching behaviour, but the prompt_cache_key field is still sent.

There is no configuration to disable it:

  • provider.nvidia.options.cache = "none" (V1) → still sent.
  • providers.nvidia.models.<id>.settings.cache = "none" (V2) → still sent.
  • The cache option is not part of the config schema.

Workaround used temporarily (and removed): a local reverse proxy that strips prompt_cache_key (also prompt_cache_retention, safety_identifier, store) before forwarding to NVIDIA. That made all affected models work, confirming the parameter is the sole cause.

Reproducible: yes, consistently, for all affected models.

Activity

  1. github-actions commented on Sep 15, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

  2. thdhyan commented on Sep 23, 2026

    @thdhyan

    Correction to my comment above — I should have checked source before "confirming." Retracting the strong claim.

    I only actually observed the Body is disturbed or locked crash, which turned out to be 100% caused by my own local plugin (a session.hook("http.request", ...) workaround written against #45113/this issue) calling request.text() without cloning first, then rebuilding a Request that still referenced the original disturbed stream. That's a bug in my local plugin code, not opencode.

    I never actually saw NIM return the prompt_cache_key 400 in my logs. Checking dev HEAD just now: packages/llm/src/providers/openai-compatible.ts routes through openai-compatible-chat.ts → openai-chat.ts, and that protocol's lowerOptions() only emits store and reasoning_effort — it never attaches prompt_cache_key/promptCacheKey to the Chat Completions body at all. So on current dev, this specific root cause looks like it may already be structurally gone (via the provider-dispatch rewrite into @opencode-ai/llm), unless there's another dispatch path I'm missing for statically-configured openai-compatible providers. Worth a maintainer double-check against a fresh 2.0.x build before anyone spends time on a PR here — I couldn't reproduce the 400 itself, only my own plugin's crash.

  3. rekram1-node commented on Oct 5, 2026

    @rekram1-node
    Contributor

    #49121 (#49121)

    Any version 2.0.4 or later has fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions