applyCaching called too freely, causes LiteLLM to fail #6473

Activity

1. RingOfStorms

   added

   bugSomething isn't workingSomething isn't working

   on Dec 30, 2025
2. github-actions

   assigned

   rekram1-node

   on Dec 30, 2025
3. github-actions commented on Dec 30, 2025

   

   github-actionsBot

   on Dec 30, 2025 – with GitHub Actions

   Contributor

   More actions

   This issue might be a duplicate of existing issues. Please check:

   • Extra inputs are not permitted, field: 'promptCacheKey' #4386: Extra inputs are not permitted, field: 'promptCacheKey' - Similar issue where cache-related fields are being added to OpenAI-compatible providers that don't support them
   • [FEATURE]: Anthropic (and others) caching improvement #5416: [FEATURE]: Anthropic (and others) caching improvement - Related discussion about caching behavior across different providers

   Feel free to ignore if none of these address your specific case.
4. rekram1-node commented on Dec 30, 2025

   

   rekram1-node

   on Dec 30, 2025

   Contributor

   More actions

   I think litellm has a flag to drop unsupported parameters, dmed person that wrote that code originally
5. RingOfStorms commented on Dec 31, 2025

   

   RingOfStorms

   on Dec 31, 2025

   Author

   More actions

   For anyone else hitting this, this plugin will work (for now at least) since the model.api.id is used primarily before this event hook is called so if we change it during this stage the applyLogic comes after.

   This could break something in the future but it works as a patch for me now.

   import type { Plugin } from "@opencode-ai/plugin";
   
   export const PatchCachingBug: Plugin = async () => {
     return {
       async "chat.params"(input) {
         const model = input.model;
         // Only touch the specific bad combo: OpenAI-compatible / gateway model
         // whose api.id contains "claude" but is NOT actually Anthropic.
         if (
           model.providerID !== "anthropic" &&
           model.api.npm !== "@ai-sdk/anthropic" &&
           typeof model.api.id === "string" &&
           (model.api.id.includes("claude") || model.api.id.includes("anthropic"))
         ) {
           // Strip the "claude" substring so ProviderTransform.message
           // does not enter the Anthropic caching branch.
           model.api.id = model.api.id.replace(/claude/gi, "cl4ude");
         }
       },
     };
   };

   I think litellm has a flag to drop unsupported parameters, dmed person that wrote that code originally

   I don't have full control over this particular LiteLLM instance myself so I can't easily get this option added yet.

   ❤️React with ❤️4Reacted by Nicolas Iderhoff, Arvind Prasad, fire and Matt Flowers
6. RingOfStorms commented on Jan 22, 2026

   

   RingOfStorms

   on Jan 22, 2026

   Author

   More actions

   Well the above plugin no longer works, either the execution order changed or something. Pretty annoying, don't feel like trying to fix it again.

   Upstream litellm has not fixed this yet either with drop params. I've resorted to running a full litellm proxy locally just to alias the names of the models that cause issues. Litellm team said they added this to their backlog so just waiting for it to get fixed there.
7. rekram1-node commented on Jan 22, 2026

   

   rekram1-node

   on Jan 22, 2026

   Contributor

   More actions

   Ah, the plugin still runs it's the code that changed internally that makes it not take affect
8. hendem commented on Jan 22, 2026

   

   hendem

   on Jan 22, 2026

   More actions

   I'm seeing the same issue: any custom model whose id contains "claude"/"anthropic" triggers applyCaching and injects cache_control into messages, which LiteLLM/OpenAI-compatible proxies reject (400 invalid_request_error / LiteLLM General Error).\n\nThis also makes a plugin workaround impossible now because the caching injection happens inside ProviderTransform.message() before any accessible plugin hooks run (chat.params runs later and experimental.chat.messages.transform can't affect the model id). So a plugin can't prevent applyCaching or strip cache_control early enough.\n\nNet: LiteLLM/OpenAI-compatible usage is broken for these model ids unless you rename/alias the model or patch core.
9. rekram1-node commented on Jan 22, 2026

   

   rekram1-node

   on Jan 22, 2026

   Contributor

   More actions

   Yeah I think we need some better approach internally, note that a plugin can still strip them out, it's kinda weird but you can override the fetch implementation for any provider and it could strip out the things, example fetch being overriden:
   

   opencode/packages/opencode/src/plugin/codex.ts

   Line 376 in c3f393b

   async fetch(requestInput: RequestInfo | URL, init?: RequestInit) {

   Ofc this isnt ideal solution just providing work around
10. hendem commented on Jan 22, 2026

    

    hendem

    on Jan 22, 2026

    More actions

    Thanks for the suggestion — that makes sense. I'll try the fetch-override approach and report back.
11. hendem commented on Jan 22, 2026

    

    hendem

    on Jan 22, 2026

    More actions

    Hey @rekram1-node, thanks for the fetch-override suggestion — it worked for me.

    I had to hook the provider ID that actually owns the OpenAI-compatible/LiteLLM proxy (redacted below), and make sure auth exists for that provider so the loader runs. After that, stripping cache_control before the request goes out fixed the 400s.

    Here’s the working plugin (provider name redacted):

    import type { Plugin } from "@opencode-ai/plugin";
    
    const STRIP_KEYS = new Set(["cache_control", "cacheControl", "promptCacheKey"]);
    
    function scrub(value: any): void {
      if (!value || typeof value !== "object") return;
      if (Array.isArray(value)) {
        for (const item of value) scrub(item);
        return;
      }
    
      for (const key of Object.keys(value)) {
        if (STRIP_KEYS.has(key)) {
          delete value[key];
          continue;
        }
        scrub(value[key]);
      }
    }
    
    function decodeBody(body: RequestInit["body"]): string | null {
      if (typeof body === "string") return body;
      if (body instanceof Uint8Array) return new TextDecoder().decode(body);
      if (body instanceof ArrayBuffer) return new TextDecoder().decode(new Uint8Array(body));
      return null;
    }
    
    export const LiteLLMFetchScrub: Plugin = async () => {
      return {
        auth: {
          provider: "redacted-classified-company",
          async loader(getAuth) {
            const auth = await getAuth();
            if (!auth) return {};
    
            return {
              async fetch(requestInput: RequestInfo | URL, init?: RequestInit) {
                if (!init?.body) return fetch(requestInput, init);
    
                const bodyText = decodeBody(init.body);
                if (!bodyText) return fetch(requestInput, init);
    
                try {
                  const payload = JSON.parse(bodyText);
                  scrub(payload);
                  const nextInit: RequestInit = {
                    ...init,
                    body: JSON.stringify(payload),
                  };
                  return fetch(requestInput, nextInit);
                } catch {
                  return fetch(requestInput, init);
                }
              },
            };
          },
          methods: [],
        },
      };
    };
    
    export default LiteLLMFetchScrub;

    ❤️React with ❤️1Reacted by Matt Flowers
12. ririnto commented on Feb 4, 2026

    

    ririnto

    on Feb 4, 2026

    More actions

    use other model id and re-mapping in config.yaml(litellm).
13. RingOfStorms commented on Mar 3, 2026

    

    RingOfStorms

    on Mar 3, 2026

    Author

    More actions

    I will close this out for now as I agree it is sort of not an opencode issue directly and is more an issue with so many different providers and proxies handling calls and payloads differently
14. RingOfStorms

    closed this as completedon Mar 3, 2026
15. soray42

    added a commit that references this issue on Mar 13, 2026

    fix: empty tool-result content and cache control level for custom @ai…

    ...

    This issue will close once commit 2d82524 is merged into the 'dev' branch.

    2d82524