Keep an idle Claude session's prompt cache warm (opt-in)

[JDProfresh]

When I step away from a Claude session for more than an hour, my next prompt re-writes the whole conversation into the cache. On a subscription the main conversation cache lives for one hour after the last request, and a cold re-write is charged at twice the input rate, while a warm read is a tenth. I measured my own last 14 days of transcripts (82 sessions): 255 gaps of 5 to 60 minutes all stayed warm, but 25 of the 27 gaps over an hour went cold and re-wrote 3.4M tokens between them. Six of those were overnight, which I don't think is worth chasing. The rest were the 1 to 4 hour gaps of a normal day: lunch, a meeting, working in another session.

Claude Code has no built-in keep-alive (there are feature requests for one in its tracker). But since 2.1.251 its statusline JSON reports prompt_cache (warm, ttl, expires_at, recache_tokens_if_cold, misses, last_miss_cause), and Codeman's statusline exporter already posts that JSON to /api/status-telemetry. So Codeman knows exactly when each session's cache will expire.

Proposal, in two steps

1. Show the cache state.

• StatusTelemetrySchema keeps the prompt_cache object instead of stripping it.
• Session.promptCache rides toState(), with a change-only session:promptCache event.
• Session Options → Respawn gets a Prompt cache readout ("Warm until 10:31 PM (1h cache). About 50k tokens would be re-read if it goes cold.").
• Users without their own statusline get cache:until HH:MM / cache:cold in Codeman's footer.

Display only, no behaviour change.

2. An opt-in keep-alive.

• Five minutes before expires_at, Codeman sends one short turn through writeViaMux: Codeman cache keep-alive: no action needed, reply with just "ok". That read resets the cache timer.
• Off by default.
• Per-session toggle on the Respawn tab next to auto-resume, plus an app-wide default and a cap under App Settings → System → Automation.

The scope decisions, all deliberate:

• Capped, by default at 3 hours after the last real prompt (configurable). The payoff is in the 1 to 4 hour gaps, and a session left overnight should go cold.
• Only on a 1-hour cache. On a 5-minute cache (API key, cloud providers, extra-usage credits) a nudge every four minutes costs more than the miss it prevents.
• Only when worth it: at least 20k tokens of context.
• Never steps on anything:
  • not while the session is working, a dialog is open, the composer holds text, or someone typed in the last three minutes;
  • not while respawn, Ralph or an agent drives the session, or during a usage-limit pause;
  • a nudge that would land too close to expiry is skipped.
• Quiet: the idle item the nudge's turn produces is opened already acknowledged, so no alert fires. Respawn's submit stamp is untouched.
• Survives a restart: the last cache report is restored, so the timer re-arms.

The arithmetic: a nudge re-reads the context at a tenth of the input rate, while a miss re-writes it at twice the rate. So a capped run of up to three nudges in a gap costs a fraction of the one miss it prevents. On a subscription both come out of plan usage rather than dollars, but the ratio is the same.

Does it work

I ran it end to end on an isolated instance with Claude Code 2.1.280:

• It armed five minutes before the reported expiry, and re-armed after a server restart.
• The nudge fired at the due second, and Claude replied "ok" in 3 seconds.
• A real prompt seven minutes past the original expiry read 112,264 tokens from cache with zero misses. An earlier un-nudged control in the same conditions re-wrote about 90k.

Both steps are built and tested, and I've been running them on my own install since 2026-10-07. The toggle and readout live on the Respawn tab, which is why I hit the bug in #549 (fix in #550).

Questions

1. Is this something you'd want in Codeman at all, or would you rather keep it out (it sends turns on its own, even if opt-in and capped)?
2. Would you take step 1 on its own first? It stands alone as a readout.
3. Is the Respawn tab the right home for the toggle, and is the nudge text OK?

If it's a yes, I'll send step 1 as a PR referencing this thread, then step 2.

[Ark0N]

Hey @JDProfresh, yes! Love this one, and the numbers you brought made it easy to say yes.

Short answers first:

1. Yes, I want this in Codeman. Opt-in and capped is the right shape.
2. Yes, send step 1 as its own PR first. It stands on its own.
3. Respawn tab is fine as the home for now, and the nudge text is fine too.

One thing changed under us though, and it matters for step 2. Over the last few days I noticed Claude Code compacting my idle sessions by itself. So I dug in, and then had a Codeman worker session go much deeper: the CC changelog and docs, the shipped 2.1.294 to 2.1.296 binaries, and 30 days of my own transcripts (48,770 API requests). Here's what came out.

Claude Code is quietly testing "idle compaction"

It's not in the changelog and not in the docs. It's in the binary, behind a server flag (tengu_sunny_locket: mode off / log / compact, plus minTokens and fireAtFraction). Read from minified code, so take the details with a little salt:

• It fires at the last request's start plus 90% of the TTL, so 54:00 after the last request on a 1h cache.
• Only on the 1h cache, only in interactive sessions, only with 200k+ tokens of context. CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS can move that floor, never below 100k.
• It stands down if a newer request happened, the cache already lapsed, the model or advisor changed, you're near a usage limit, a turn is running, someone interacted in the last 60 s, or another process holds the session.
• It compacts while the cache is still warm, so the summary request reads from cache. Then it adds the notice "Compacted while idle, before the prompt cache expired".
• Opt-out: "idleCompaction": false in settings. The schema says setting it to true does not turn it on, the server decides. Turning auto-compact off kills it too.

What it did here: 13 idle compactions between Oct 8 ~17:00 and Oct 9 05:35 UTC, on builds 2.1.289 to 2.1.295. Every one landed 54.7 to 55.7 min after the last reply. The 6 times I came back, the first turn re-wrote 19k to 106k tokens instead of 440k to 884k. The other 7 were never touched again, so for those it was pure (small) cost.

Then it stopped. After Oct 9 05:35, four eligible gaps (214k to 487k of context, same builds) went fully cold. The cached flag in ~/.claude.json flipped between compact and off for a few minutes while different sessions refreshed it, and it has read off ever since. Looks like an experiment. Nobody can count on it right now.

There's also a native keep-alive in the binary, switched off

So "Claude Code has no built-in keep-alive" is true for users today, but there's one in the binary behind a flag that's currently off: tengu_sequential_cloud (false here), endpoint POST /v1/messages/cache_touch, beta cache-keepalive-2026-09-03. Same salt as above, the numbers are code defaults, not observed behaviour. When active it switches the main conversation to the 5 minute TTL and touches the cache every 30 s, but only within a 60 minute activity window. So it's a cheaper version of today's 1h behaviour, not a multi-hour keep-alive. And since it moves the TTL to 5m, idle compaction never arms while it's on.

The cold re-writes are real, and big

From the 30 days of transcripts:

• Of 158 returns after a gap of 60+ min, 156 went cold. That re-wrote 65.9M tokens, 38.9% of all main-conversation cache writes.
• The 1h cache itself works great: of 1,262 gaps between 5 and 60 min, 2 went cold.
• Almost nothing else causes misses. Zero misses from model switches, effort changes, version changes, MCP or OAuth. Gaps over an hour are the whole story.
• My contexts skew big (1M sessions): 140 of the 158 cold returns were 200k+. Yours averaged about 136k, so most of yours sit below the idle-compaction floor. Different fleets, same problem.

One correction to the arithmetic: on Opus 5.5 and Sonnet 5.5 cache reads are 0.05x, not 0.1x (footnote on the pricing page). The 1h write is still 2x. So nudges are even cheaper than you wrote.

Step 2 as written collides with idle compaction

Both clocks start at the same moment: expires_at is the last request's dispatch time plus the TTL. CC fires at 54:00, your nudge at 55:00. So CC always goes first, by 60 s, whenever the flag is on. The compactions took 53 to 109 s (11 of 13 over a minute), so the nudge usually arrives mid-compaction. Either it gets dropped by your busy guard or it runs on the summary. Either way the full context is already gone, and the nudge only keeps the summary warm.

My first idea was "just nudge earlier", but that isn't stable. fireAtFraction is server-tunable down to 0.5 (fires at 30:00), and a nudge before CC fires is a newer request that re-arms CC's timer from the nudge. That's fine while it works, but we'd be racing a number we can't see.

What I'd suggest instead: Codeman owns the idle policy for sessions where the keep-alive is on.

• Codeman passes "idleCompaction": false inside its one merged --settings JSON (we already build exactly one there for the statusLine exporter, advisor and ultracode). Then CC's flag can't pre-empt us, whatever the server says. Gate it on the CLI version.
• One catch: --settings is fixed at launch. So that covers sessions started with keep-alive on, but flipping the toggle on a session that's already running can't change it, and relaunching to apply it costs exactly the re-write we're trying to avoid. The thing to check is the case's settings.local.json (the route the model default already uses). I haven't verified that CC picks idleCompaction up from there live, and that file is per case, not per session. Worth a quick test before building the toggle around it.
• Keep warm exactly like you proposed: nudge at expires_at - 5 min, 1h TTL only, all your guards, capped. The sim below says a 2h cap is slightly cheaper than 3h.
• When the cap runs out, send /compact shortly before the last expiry instead of letting it go cold. That way the session never re-writes its full context on return. But only at 100k+ tokens: below that, a compaction plus the post-compaction re-write costs more than just going cold.
• Skip agent workers that already reported done. They're most of the "never came back" tails, which is where the keep-alive wastes money.
• If a session ever reports ttl: "5m" with the native keep-alive active, stand down. CC is in charge then.

Policy simulation over my real 30 days of gaps (relative to doing nothing)

166 returns after 55+ min gaps, plus sessions that never came back (counted as an upper bound). API-equivalent prices, which are only a proxy for plan usage. How subscription limits weigh reads vs writes isn't documented. Nudge output assumed at 300 tokens.

Policy	Cost vs doing nothing	Full context kept on return
Nothing (cold on return, today with the flag off)	100%	yes, but slow and expensive
CC idle compaction alone (flag on)	28%	no, 145 returns compacted
#554 as proposed, idle compaction off	65%	within the cap, cold after
#554 as proposed, idle compaction on (the collision)	42%	no, 145 compacted anyway
Codeman-owned: keep warm 3h, then /compact	30%	yes on 65 more returns than CC's
same with a 2h cap	28%	yes on 54 more returns than CC's

Caveat: on xhigh effort with thinking on, a nudge might output ~3k tokens instead of 300. Then the 3h keep-warm row goes to 38% and #554 alone to 74%. Worth measuring a real nudge before picking defaults.

Asks for step 1

• Keep the whole prompt_cache object, and make it .nullish(). expires_at, hit_ratio, last_miss_at and recache_tokens_if_cold are all documented as possibly null, and Zod's .optional() rejects null (that one has bitten us twice).
• Show warm until / cold, recache_tokens_if_cold, hit_ratio and last_miss_cause.causes. "ttl_expired_1h" vs "tools_changed" is exactly the hint people need.
• The statusline re-runs when a warm cache hits expires_at (documented), so the flip to cold reaches /api/status-telemetry without any timer on our side. From the code, fork requests that read the main cache (compaction, /recap) also push expires_at forward.
• recache_tokens_if_cold goes null right after a compaction and expected_rebuilds goes up. Use that to show "compacted at 10:31" and to make step 2 stand down for that gap. Don't raise an alert for CC's own idle compaction.
• Persist the last value per session so it survives a Codeman restart (you already planned this for the timer).

What the release notes do document

• 2.1.84: idle-return prompt nudging /clear after 75+ min. 2.1.92: Pro footer hint on return after the cache expired.
• 2.1.108: ENABLE_PROMPT_CACHING_1H / FORCE_PROMPT_CACHING_5M. 2.1.129: fixed the 1h TTL silently dropping to 5 min.
• 2.1.243: promptCacheTtl / subagentPromptCacheTtl (env CLAUDE_CODE_PROMPT_CACHE_TTL).
• 2.1.251: prompt_cache for statusline scripts and /cost. 2.1.260: last_miss_cause.
• A long run of cache-miss fixes from 2.1.248 to 2.1.281 (OAuth refresh, advisor background requests, model switches, resume). Being on a current build matters.
• Docs: a subscription that goes over its limit and draws on usage credits drops the main conversation to the 5 min TTL. Your "1h only" check covers it, as long as it reads ttl fresh each time.

Other things the research turned up on our side

• The advisor reads the whole conversation uncached on every call (documented on the advisor page). In the 4 days I leaned on it, that was about 10% of my 14-day usage. Those reads show up in usage.iterations in the transcript, not in the top-level token counts, so they're easy to miss.
• A few latent cache killers in Codeman itself, all off by default and not firing here, that I'll fix on our side:
  • respawn's skip-clear check assumes a 200k window, so on a 1M session it only skips /clear below 60k instead of 300k;
  • the auto-compact token counter never goes down after a /compact;
  • the orchestrator's between-phase /compact is sent without \r, so it never submits.
• Exposing promptCacheTtl per session would help anyone on usage credits a lot. In my data, the 5-55 min gaps re-written at a 5m TTL would cost far more than the cheaper writes save.

So: go ahead with step 1 whenever you're ready and reference this thread. For step 2, I'd love the "Codeman owns the idle policy" shape (keep warm, then /compact, with idleCompaction: false while it's on). Happy to talk through any of it here first.

Thanks a lot for this, and for #550, which shipped in 1.40.0!

[JDProfresh]

Thanks for digging into this, and for the 30 days of data. The idle-compaction find changes step 2 in a good way, and I'm glad step 1 lands on its own.

Step 1 is up as its own PR, #607. It's just the readout plus the schema asks you listed (keep the whole prompt_cache object, .nullish(), show warm-until/cold, recache_tokens_if_cold, hit_ratio, last_miss_cause.causes, persist per session, don't alert on CC's own idle compaction).

On step 2, yes, let's do the "Codeman owns the idle policy" shape. Concretely that means: idleCompaction: false goes into the one merged --settings JSON at launch, version-gated; keep warm with the nudge at expires_at - 5 min, 1h TTL only, all the guards, 2h cap per your sim; when the cap runs out send /compact shortly before the last expiry, but only at 100k+ tokens; skip agent workers that already reported done; and stand down if a session ever reports ttl: 5m with the native keep-alive active.

I pulled apart the installed binary (2.1.294, same family you read) on the live-pickup question, since that's the one that decides how the toggle behaves. The answer is that it does not pick up live, and the reason is in the resolver. idleCompaction is read through the memoized merged-settings store (getSettingsForSource cached, getSettingsForSourceUncached as the explicit bypass), and the idle-compaction eligibility check reads that cache at fire time. The cache is invalidated when Claude Code writes a settings file itself, when plugins reload, and on managed-settings, account, or sync changes. I could not find any settings-file watcher that wires an external write to settings.local.json into that invalidation. So an outside process writing the case's settings.local.json on a running session is not reliably seen until some unrelated invalidation happens. The source precedence is user < project < local < flag < policy, resolved high to low, so the --settings flag outranks settings.local.json and everything below it, and only managed/policy settings outrank the flag. That makes the launch flag authoritative, which is what we want.

The design consequence: a session started with keep-warm on gets idleCompaction: false via --settings and fully owns the policy, flag or no flag on the server. A session toggled on after launch can't install the opt-out without a relaunch, and a relaunch costs exactly the re-write we're trying to avoid. So for the after-launch case I'd keep it best-effort: we nudge and keep the cache warm, but while the server flag is on, CC can still compact first (it fires at 54:00, our nudge at 55:00, the collision you measured). I'll run your settings.local.json test to confirm before I wire the toggle, but the code path already predicts no live pickup, so I'm not planning to build the toggle around it.

On the nudge cost you flagged as worth measuring: I've had step 2 running on my own install, so I have real nudges to read. Three of them fired on one Opus 5.5 session in normal mode (Oct 8 to 9), and every one output exactly 4 tokens ("ok") with 0 thinking tokens, replied in 2 to 4 seconds, read the full 243k to 333k of context from cache, and created only 56 to 434 new cache tokens. So observed is 4 output tokens, not the 300 your sim assumed, which makes the keep-warm rows cheaper than the table shows. The one thing I can't close from my own data: all three were Opus 5.5 where thinking came out at 0, and I don't have a nudge captured on xhigh with thinking forced on, so I can't fully rule out your ~3k worst case there. A one-word directive produced zero thinking even on Opus 5.5, which makes me doubt it, but I'd keep 300 as the conservative default until I measure an xhigh case.

For what it's worth, every binary number in your write-up held on 2.1.294: fireAtFraction default 0.9 (so 54:00) with a 0.5 floor, minTokens default 200k with a 100k floor via CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS, 1h only, mode off/log/compact, the schema text verbatim ("Setting it to true does not turn idle compaction on"), and the native keep-alive (tengu_sequential_cloud off, /v1/messages/cache_touch, touch every 30s, 60 min window, beta cache-keepalive-2026-09-03). Nothing to correct.

Two things where your answer changes what I build:

1. Scope of idleCompaction: false. Only on keep-warm-on sessions, or default it off for every Codeman Claude session? Idle compaction is an unannounced experiment that flips on and off under us, and it conflicts with Codeman owning the conversation lifecycle, so there's a case for turning it off everywhere we launch. That decides whether the launch flag is conditional or always on.

2. For a session toggled on after launch, are you OK with best-effort (keep warm, but CC may still idle-compact first while its flag is on) rather than forcing a relaunch that costs the re-write we're avoiding?