Cache hit rate is low / zero #14065

Activity

1. surapuramakhil

   added

   bugSomething isn't workingSomething isn't working

   on Feb 18, 2026
2. github-actions

   assigned

   rekram1-node

   on Feb 18, 2026
3. github-actions

   added

   coreAnything pertaining to core functionality of the application (opencode server stuff)Anything pertaining to core functionality of the application (opencode server stuff)

   on Feb 18, 2026
4. github-actions commented on Feb 18, 2026

   

   github-actionsBot

   on Feb 18, 2026 – with GitHub Actions

   Contributor

   More actions

   This issue doesn't fully meet our contributing guidelines.

   What needs to be fixed:

   • Steps to reproduce: Please provide detailed steps showing how to reproduce the cache hit rate issue
   • System information: Please fill in your Operating System and Terminal environment
   • Additional context: Explain what cache hit rate you're expecting vs what you're seeing, and any relevant configuration details

   Please edit this issue to address the above within 2 hours, or it will be automatically closed.

   ---

   This issue might be a duplicate of existing issues. Please check:

   • Due to the lack of userId and sessionId, some models will not hit the cache. #8623: Due to the lack of userId and sessionId, some models will not hit the cache (directly related to cache hit problems)
   • /compact doesn't utilize prompt caching #10342: /compact doesn't utilize prompt caching (related cache utilization issue)
   • applyCaching called too freely, causes LiteLLM to fail #6473: applyCaching called too freely, causes LiteLLM to fail (related caching issue)

   If you believe this was flagged incorrectly, please let a maintainer know.
5. github-actions

   added

   needs:complianceThis means the issue will auto-close after 2 hours.This means the issue will auto-close after 2 hours.

   on Feb 18, 2026
6. rekram1-node

   removed

   needs:complianceThis means the issue will auto-close after 2 hours.This means the issue will auto-close after 2 hours.

   on Feb 18, 2026
7. bhagirathsinh-vaghela commented on Feb 23, 2026

   

   bhagirathsinh-vaghela

   on Feb 23, 2026

   Last edited by bhagirathsinh-vaghela

   More actions

   PR #14743 addresses the root causes of Anthropic prefix cache misses: system prompt was a single merged block (dynamic content invalidated stable content), bash tool schema included the absolute cwd (changed per-repo), and skill tool ordering was nondeterministic. If you're on Anthropic, cross-repo first-prompt hit rate goes from 0% → 97.6% in my setup. The prefix stability fixes may also help providers with automatic prefix caching (OpenAI, DeepSeek, Gemini, etc.) but I can't confirm for DeepInfra specifically.
8. surapuramakhil commented on Feb 23, 2026

   

   surapuramakhil

   on Feb 23, 2026

   Author

   More actions

   for now I have switched to gemini Api's - As i am able to see chache hits for it.

   👍React with 👍1Reacted by Bhagirathsinh Vaghela
9. TomLucidor commented on Feb 24, 2026

   

   TomLucidor

   on Feb 24, 2026

   More actions

   Does this problem apply to other LLM APIs? #5224
10. surapuramakhil commented on Feb 24, 2026

    

    surapuramakhil

    on Feb 24, 2026

    Author

    More actions

    @TomLucidor

    #5224 -> i have seen this in deep infra

    but suprisingly gemini Api's are getting cache chits but not deep infra, both uses byte idential prefix (implicit caching)
11. surapuramakhil commented on Feb 25, 2026

    

    surapuramakhil

    on Feb 25, 2026

    Last edited by surapuramakhil

    Author

    More actions

    @thdxr @adamdotdevin @rekram1-node

    request to have a look it, I have inspected OpenCode SQLite DB -> it confirms gemini "byte identical prefix cache is working for gemini but its not working for deepInfra"
12. surapuramakhil commented on Feb 25, 2026

    

    surapuramakhil

    on Feb 25, 2026

    Author

    More actions

    note: looks like this is only happening to moonshoot/ kimi 2.5 model
13. rothnic commented on Apr 2, 2026

    

    rothnic

    on Apr 2, 2026

    More actions

    Adding one more data point from a separate investigation that may help narrow this down.

    In our case, we saw repeated delayed turns with inconsistent cache.read, but the strongest signal was that multiple delayed turns shared the same stored request-state snapshot while cache behavior still flipped between hit and miss.

    An anonymized example from the analyzed session tree:

    Time	Snapshot stable?	Input tokens	Cache read	Result
    17:41:09	yes	81,851	0	tool-calls
    17:41:17	yes	82,132	0	tool-calls
    17:41:27	yes	82,859	0	tool-calls
    17:41:40	yes	196	83,392	tool-calls
    17:41:53	yes	-423	84,160	tool-calls
    17:42:09	yes	84,358	0	tool-calls

    Because the stored snapshot stayed stable inside the cluster, this did not look like a simple case of the system prompt changing on every miss. It seems more consistent with either cache reuse instability for effectively identical request state, or inconsistent token/cache accounting.

    Related issue with the fuller anonymized analysis: #20565
14. thdxr

    removed

    bugSomething isn't workingSomething isn't working

    coreAnything pertaining to core functionality of the application (opencode server stuff)Anything pertaining to core functionality of the application (opencode server stuff)

    on May 3, 2026
15. rekram1-node commented on May 22, 2026

    

    rekram1-node

    on May 22, 2026

    Contributor

    More actions

    believe this is resolved now

    👍React with 👍1Reacted by TomLucidor
16. rekram1-node

    closed this as completedon May 22, 2026