App-server sessions need stable prompt-cache key support for identical startup context

What version of Codex CLI is running?

0.129.0

Which model were you using?

gpt-5.3-codex-spark

What platform is your computer?

Windows, using Codex app-server through an external multi-session orchestrator.

What issue are you seeing?

Independent fresh Codex app-server sessions do not appear to get full first-turn prompt-cache reuse even when the client sends the same stable startup context byte-for-byte.

From local instrumentation in a multi-session app-server workflow:

Session A first turn: 11,749 input tokens, 6,528 cached input tokens (~55.6%).
Session B first turn: 11,749 input tokens, 6,400 cached input tokens (~54.5%).
The model-visible first-turn startup prefix sent by the client was byte-identical across both sessions.
Both sessions were independent fresh Codex app-server threads with different Codex thread IDs.

The likely cause is that Codex currently derives prompt_cache_key from the unique thread id:

codex/codex-rs/core/src/client.rs

Lines 713 to 728 in 872b8b1

    
           let prompt_cache_key = Some(self.state.thread_id.to_string()); 
        
           let service_tier = 
        
               service_tier.filter(|service_tier| model_info.supports_service_tier(service_tier)); 
        
           let request = ResponsesApiRequest { 
        
               model: model_info.slug.clone(), 
        
               instructions: instructions.clone(), 
        
               input, 
        
               tools, 
        
               tool_choice: "auto".to_string(), 
        
               parallel_tool_calls: prompt.parallel_tool_calls, 
        
               reasoning, 
        
               store: provider.is_azure_responses_endpoint(), 
        
               stream: true, 
        
               include, 
        
               service_tier, 
        
               prompt_cache_key,

That is reasonable for cache reuse within a single long-lived thread, but it prevents app-server clients from intentionally sharing cache for stable startup/base context across multiple independent sessions. This is especially visible for multi-session / multi-agent orchestrators that start several fresh Codex sessions with identical base instructions, developer instructions, project context, and first-turn setup.

I did not find a current app-server option or schema field that lets the client provide a stable prompt cache key or split the cache key for the reusable startup prefix.

This is related to, but more specific than, the broader low-cache-hit issue in #20301. That issue appears to cover general GPT-5.5 cache-hit anomalies; this report is about the app-server architecture using per-thread cache keys for otherwise identical fresh sessions.

What steps can reproduce the bug?

Start two independent fresh Codex app-server sessions for the same project.
Use the same model and runtime configuration for both sessions.
Send the same startup/base/developer/project context in the first turn of each session.
Confirm locally that the model-visible first-turn prefix is byte-identical across the two sessions.
Compare input_tokens and cached_input_tokens for the first completed turn of each session.

In the observed case, both sessions had exactly 11,749 input tokens, but only about 55% of input tokens were reported as cached, despite the stable prefix being byte-identical.

What is the expected behavior?

App-server clients should have a way to opt into stable prompt-cache reuse for stable startup/base context across independent fresh sessions.

Possible fixes:

expose a client-provided prompt_cache_key / cache namespace for app-server sessions;
allow app-server clients to provide a stable cache key for base/developer/startup context while keeping per-thread state isolated;
or otherwise avoid tying reusable startup-context caching exclusively to the unique thread id.

The important property is that two independent sessions with identical stable startup context can share cache for that prefix without sharing conversation state.

Additional information

A previous PR appears to have explored passing a prompt cache key when starting Codex, but it was closed without merging: #5949.

Related cache-key propagation work for compaction: #21249.

Related fork prompt-cache-state work: #20909.

If maintainers want private thread/session IDs or uploaded feedback bundles for the two observed sessions, I can provide them separately. I avoided putting those IDs in this public issue body.

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

Low cache hit rate when Codex integrates with GPT-5.5 #20301

Powered by Codex Action

FWIW, I independently saw the same kind of instability from app-server instrumentation.

In one run with byte-identical reusable context, I saw 25,472 / 25,657 input tokens cached, so about 99.3%. In another comparable run, I saw only 2,432 / 25,657, so about 9.5%. Forked-thread runs were similarly inconsistent: one child turn got 15,744 / 25,327 cached (62.2%), another only 2,432 / 25,327 (9.6%).

So this does not look like just noisy accounting. The cache can clearly work very well, but app-server clients currently cannot make it reliable for identical startup/fork context because the cache key is tied to unique thread ids.

A promptCacheKey/cache namespace option in app-server, or using the shared root/session id for fork families, would be very useful here with a big payoff for multi-agent orchestrators.

codex: the models it serves, and one live thread instead of a fresh o…

Adding production numbers from a multi-session orchestrator, since this hits us hard on the ChatGPT-authenticated backend.

Setup: OpenClaw drives the Codex app-server (codex 0.153.4, gpt-6-astra, ChatGPT login, WebSocket transport) for an internal assistant. Every new conversation, scheduled job and helper run is a fresh app-server thread, and the first request of every thread carries the same ~42k-token prefix (base instructions, ~20 dynamic tool schemas, developer instructions).

Measured over three days (Oct 3–5, 390 fresh threads):

349 first requests (89 %) had cached_input_tokens = 0.
27 had ~11–12.4k cached (roughly the tool-schema prefix), 14 had more than 20k cached — apparently the few that happened to be routed to a warm machine.
On a busy weekday (~300 thread starts), thread starts were ~36 % of all uncached input plus output, which is what our weekly usage tracks.
Within a thread, caching works well: after pauses of up to an hour, ~90 % of the previous context is still served from cache.

What we found in the source: since #44862 the cache affinity follows the session-id header (the root thread id), and only ephemeral root forks inherit the parent's affinity. Ephemeral threads are not persisted, so this does not help conversations that have to be resumed later (see also #44716).

What would solve it for us (either would do):

a cache-affinity / promptCacheKey option on thread/start and thread/fork that is applied to both prompt_cache_key and the session-id header, or
letting persistent forks opt into the parent's affinity (e.g. inheritCacheAffinity: true), so clients can start conversations from a warm template thread.

We would shard such keys ourselves (per agent, hash mod N) to stay below the ~15 requests/minute overflow guidance. Happy to share more data.

※

Adding a controlled measurement on the ChatGPT-authenticated backend that isolates the cause and shows a partial workaround.

Setup: codex-cli 0.162.0-alpha.17.2, app-server, gpt-6.1-sol (low effort), ChatGPT sign-in. Every chat sent the same first message on top of an identical ~24k-token startup prefix (base instructions, dynamic tools, strict output schema). Arms were interleaved (A B B A A B B A A B) within a few minutes.

Arm	First request: input	Cached input	Chats
A: `thread/fork` with `experimentalPredictionMode`, `ephemeral`, `excludeTurns: true` from a warmed base thread	23,906	23,680 (99.1%)	5/5
B: fresh `thread/start`, byte-identical prompt	23,870	0	5/5

Arm B chats repeated the exact same prompt seconds apart and never got a cache hit, so reuse follows session affinity rather than identical bytes.

Against a local fake endpoint, the same binary sends the parent thread's id as both prompt_cache_key and the session-id header on prediction-mode forks (while client_metadata keeps the fork's own ids). Plain forks, beforeTurnId forks and fresh threads send their own id in both. That matches the session-header affinity described in #44716 / #44862.

Prediction-mode forks are a workable stopgap for clients that start many short sessions with the same startup context. But they are ephemeral (a restarted app-server can't thread/resume them: "no rollout found"), they need a warm-up turn on a base thread, and the parameter is experimental.

Request: let app-server clients pass a stable cache-affinity key on thread/start (and thread/resume), used for both prompt_cache_key and the session-id header, while keeping the real session identity in metadata. Durable, resumable threads with identical startup context could then reuse the cached prefix the same way prediction forks already do.

Navigation Menu

Repository navigation

App-server sessions need stable prompt-cache key support for identical startup context #21796

What version of Codex CLI is running?

Which model were you using?

What platform is your computer?

What issue are you seeing?

What steps can reproduce the bug?

What is the expected behavior?

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	let prompt_cache_key = Some(self.state.thread_id.to_string());
	let service_tier =
	service_tier.filter(\|service_tier\| model_info.supports_service_tier(service_tier));
	let request = ResponsesApiRequest {
	model: model_info.slug.clone(),
	instructions: instructions.clone(),
	input,
	tools,
	tool_choice: "auto".to_string(),
	parallel_tool_calls: prompt.parallel_tool_calls,
	reasoning,
	store: provider.is_azure_responses_endpoint(),
	stream: true,
	include,
	service_tier,
	prompt_cache_key,

App-server sessions need stable prompt-cache key support for identical startup context #21796

Description

What version of Codex CLI is running?

Which model were you using?

What platform is your computer?

What issue are you seeing?

What steps can reproduce the bug?

What is the expected behavior?

Additional information

Activity

github-actions commented on May 8, 2026

findusl commented on Jun 11, 2026

rodja commented on Oct 6, 2026

jack-hyperbond commented on Oct 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions