Repository navigation
[FEATURE]: Anthropic (and others) caching improvement #5416
Description
Feature hasn't been suggested before.
- I have verified this feature I'm about to request hasn't been suggested before.
Describe the enhancement you want to request
I've recently started utilizing opencode, and saw my token usage was somewhat higher with the same general workflow in opencode that I was using in claude code. After doing a little research, I determined the cache model and prompt structure being used was suboptimal for claude-based models.
I've submitted a PR that attempts to address this, and allows configuration at the provider and per-agent level. Some of my workstreams may run for long periods of time, and have gaps inbetween runs for certain types of agents (for example, review agents may not run frequently, but when they do have a large amount of static context used for their instructions), so allowing TTL to be overridden at the agent level made sense to me.
I did some basic testing with the patch, and it made a significant difference in non-cached vs. cached usage, which with claude pricing, can make a huge difference in the cost of using these LLMs. Unfortunately, the minimum cache size wasn't available programmatically, so I had to build a lookup table for the various models. Basic performance/cache testing is in the PR.
I tried to create the PR in a way that wouldn't negatively impact any other models/providers, but could also be used as a starting point for other models/providers that had specific cache implementation requirements. Later this model can be extended to configuration beyond caching.
Activity
github-actions commentedon Dec 12, 2025
This issue might be a duplicate of existing issues. Please check:
The proposed solution in #5416 (TTL-based cache configuration for Anthropic) complements the broader compaction strategy discussed in #4317. Both issues aim to optimize token usage, but from different angles - #5416 focuses on cache TTL specifics for Anthropic models, while #4317 addresses core compaction infrastructure.
Feel free to ignore if this is specifically an Anthropic caching implementation that differs from the broader compaction approach.
[-][FEATURE]: Anthropic prompt caching improvement[/-][+][FEATURE]: Anthropic (and others) caching improvement[/+]konekohana commentedon Jan 15, 2026
I'm running into the same issue as well. I'm using OpenRouter for completions and I can see in their logs that the cache expires often with Anthropic models if I go away from the conversation for a few minutes. Using opencode would probably be much cheaper if I could combine it with the 1 hour TTL anthropic offers. I tried to set the header using the per model configuration, but it seems that this can only be used for string header values:
bhagirathsinh-vaghela commentedon Feb 23, 2026
PR #14743 addresses the root causes of cache misses directly: system prompt split into stable/dynamic blocks, bash tool schema cwd removal, skill tool ordering stabilization, optional system prefix freeze, and optional 1h TTL. Cross-repo first-prompt hit rate goes from 0% → 97.6% in my setup (your numbers will differ based on your tool set and global AGENTS.md size, but the cross-repo miss is eliminated regardless).
github-actions commentedon Apr 25, 2026
To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.
roadmap: #219 filed — Anthropic prompt-caching opt-in is structurally…
roadmap: #219 filed — Anthropic prompt-caching opt-in is structurally…
roadmap: #219 filed — Anthropic prompt-caching opt-in is structurally…
roadmap: ultraworkers#219 filed — Anthropic prompt-caching opt-in is …
roadmap: #219 filed — Anthropic prompt-caching opt-in is structurally…
roadmap: ultraworkers#219 filed — Anthropic prompt-caching opt-in is …
roadmap: ultraworkers#219 filed — Anthropic prompt-caching opt-in is …
roadmap: ultraworkers#219 filed — Anthropic prompt-caching opt-in is …