Skip to content

Low cache hit rate when Codex integrates with GPT-5.5 #20301

Open
Open
Low cache hit rate when Codex integrates with GPT-5.5#20301
Labels
bugSomething isn't workingrate-limitsIssues related to rate limits, quotas, and token usage reporting

Description

@pyfdtic
pyfdtic
opened on Apr 30, 2026

What version of Codex CLI is running?

0.125.0

Which model were you using?

gpt-5.5

What platform is your computer?

WSL2

What terminal emulator and version are you using (if applicable)?

Windows Terminal

What issue are you seeing?

When Codex integrates with the GPT-5.5 model, its cache hit rate is very low, which causes costs to be consumed rapidly.

In contrast, OpenCode integrated with GPT-5.5 has no such problem and maintains a high cache hit rate.

Besides, Codex paired with GPT-5.4 also achieves a high cache hit rate normally.

What steps can reproduce the bug?

We tested Codex integrated with GPT-5.4 and GPT-5.5 respectively. By checking the session logs of both versions, we can see there is a significant difference in the frequency of cached_input_tokens.

What is the expected behavior?

The expected behavior is that Codex with GPT-5.5 should maintain a normal high cache hit rate, consistent with the performance of Codex + GPT-5.4 and OpenCode + GPT-5.5, to avoid excessive and rapid cost consumption.

Additional information

No response

Activity

  1. pyfdtic
    added
    bugSomething isn't working
    on Apr 30, 2026
  2. github-actions
    added
    CLIIssues related to the Codex CLI
    rate-limitsIssues related to rate limits, quotas, and token usage reporting
    on Apr 30, 2026

  3. github-actions commented on Apr 30, 2026

    @github-actions
    github-actionsbot
    on Apr 30, 2026 – with GitHub Actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  4. etraut-openai
    removed
    CLIIssues related to the Codex CLI
    on Apr 30, 2026

  5. etraut-openai commented on Apr 30, 2026

    @etraut-openai
    etraut-openai
    on Apr 30, 2026
    Contributor

    Thanks for the feedback. We track cache miss rates closely on the server, and we don't see any anomalous behavior related to Codex and gpt-5.5. It's possible that the session you analyzed wasn't representative.

  6. etraut-openai
    closed this as not plannedon Apr 30, 2026

  7. etraut-openai commented on Apr 30, 2026

    @etraut-openai
    etraut-openai
    on Apr 30, 2026
    Contributor

    We're receiving similar reports from other sources, so I'm going to reopen this issue until we're able to reconcile these reports with our server-side stats.

  8. etraut-openai
    reopened this on Apr 30, 2026

  9. kkondaurov commented on Apr 30, 2026

    @kkondaurov
    kkondaurov
    on Apr 30, 2026

    Can confirm. Codex Desktop on Mac OS with an API key.

    Other users in our org with API keys also have this problem, although they're TUI/OpenCode users. So it's probably not about the harness.

    Had to switch back to 5.4, where caching is totally fine. On my personal subscription, 5.5 is totally fine.

    @etraut-openai I can share session ids with cache misses if it helps.

  10. hsir commented on May 2, 2026

    @hsir
    hsir
    on May 2, 2026

    Yes, I also confirm this. I am using the API Key billing, not a subscription. Whether using the CLI or the desktop version, when using GPT-5.5, the cache hit rate is almost always 0%. Once I switch back to GPT-5.4, the cache ratio returns to normal, at 90% or higher. I didn’t make any changes other than switching the model in Codex settings. I even checked the request body JSON shape, nothing wrong at all. I think there must be something wrong in OpenAI server

  11. AetherMagee commented on May 2, 2026

    @AetherMagee
    AetherMagee
    on May 2, 2026

    Can confirm too, GPT-5.5 cache hit rates are close to zero in both Codex and Opencode for me. GPT-5.4 is doing fine with otherwise identical configs.

  12. etraut-openai commented on May 2, 2026

    @etraut-openai
    etraut-openai
    on May 2, 2026
    Contributor

    If you're seeing this problem, please use /feedback to upload the session and post the session ID here. Thanks!

  13. kkondaurov commented on May 4, 2026

    @kkondaurov
    kkondaurov
    on May 4, 2026

    If you're seeing this problem, please use /feedback to upload the session and post the session ID here. Thanks!

    Hey @etraut-openai , reported these sessions via /feedback:

    019df1e4-9669-76a1-bdde-280bd0d241ac
    019df1e7-bf0a-7543-b3ec-30074cd7d528
    019df1ed-1dc4-7221-9888-e5f75488ffde

  14. ryanmadsenn commented on May 4, 2026

    @ryanmadsenn
    ryanmadsenn
    on May 4, 2026

    Also been seeing this issue. Cache hit rate with 5.4 was ~94% and it dropped to 85% with 5.5. In certain instances I saw 8 consecutive cache misses within my session.

  15. darinkishore commented on May 5, 2026

    @darinkishore
    darinkishore
    on May 5, 2026

    I uploaded several affected sessions with /feedback / feedback/upload.

    The pattern I see locally is not a steady low cache hit rate. The aggregate hit
    rate is high, but there are isolated catastrophic collapses inside otherwise
    healthy high-context sessions: 200k-240k input-token requests with 1-3% cache
    hit rate, seconds after 98-100% hits, then immediately recovering.

    Uploaded thread IDs and representative miss moments:

    • 019dda83-e45e-7740-89ec-3c6c789c398d: 2026-04-29T18:49:55.375Z, rollout line 748, input 230,330, hit 2.8%, uncached 223,802. Prior token events were 99.6/99.8/99.7/99.9/99.8/99.5/99.8/99.8%.
    • 019dda86-583a-7430-9b0f-af1a6dd3c755: 2026-04-29T19:34:29Z, rollout lines 826/839, input 229,641, hit 2.8%, uncached 223,113.
    • 019dda9a-2e7f-7af0-a9cb-bcb63cec8b77: 2026-05-03T21:59:09.713Z, rollout line 119649, input 240,110, hit 1.4%, uncached 236,654. Prior token events: 98.6/98.3/97.3/99.7/99.1/99.8/99.9/99.8%; next events recovered to 99.6/99.7/99.7/98.9/99.8% after a 10.6s gap and only 178 new chars.
    • 019def0d-178f-79d1-ad2d-5213fa7245c6: 2026-05-03T22:31:08.296Z, rollout line 8418, input 238,349, hit 2.7%, uncached 231,821. Prior token events: 99.9/99.6/99.7/99.1/99.7/99.8/99.5/99.5%; next recovered to 99.1/99.3% before compaction.
    • 019ded0c-78f5-7cf2-9645-110e2c40bf47: 2026-05-05T00:00:35.489Z, rollout line 14782, input 218,023, hit 1.6%, uncached 214,567. Prior: 94.5/99.8/99.6/97.0/94.8/99.7/99.9/98.5%; next recovered to 99.9/99.8/99.7%.

    Bulk local scan, 2026-04-29 through 2026-05-05 UTC:

    • 160 rollout files, 138,559 token events.
    • 75,142 fresh-ish token events after local replay/fork filtering.
    • gpt-5.5 fresh-ish input: 11.107B tokens.
    • gpt-5.5 fresh-ish uncached: 239.6M tokens.
    • gpt-5.5 fresh-ish hit rate: 97.8%.
    • Severe tail: 109 fresh-ish events with input >= 100k and hit < 10%.
    • Very severe tail: 28 fresh-ish events with uncached >= 200k.

    This does not look like cache expiry: several examples happen 5-30s after
    strong hits and recover immediately. It also does not look like the documented
    ">15 requests/min with the same prefix" cache-key overflow case; in the
    uncached>=200k band I found 0 events with 15+ previous same-thread token
    events in the preceding minute.

    Related live-session observation while uploading feedback: two of the long
    threads (019dda9a... and 019ded0c...) were not inert when resumed just to
    submit /feedback; they resumed/continued paused work and had to be
    interrupted again before or after upload. That may be related to the "old active
    mega threads still updating" behavior users are seeing.

  16. kkondaurov commented on May 8, 2026

    @kkondaurov
    kkondaurov
    on May 8, 2026

    Still seeing these cache misses for 5.5, which make session cost skyrocket.

    Submitted a couple more sessions via feedback, here are correlation ids:
    930e9e7b-1f1e-40ff-acad-e709a6371646
    2c22e6d4-a83a-49ed-82dd-a514936d918d

  17. Raw1mage commented on May 10, 2026

    @Raw1mage
    Raw1mage
    on May 10, 2026

    Codex Responses API — Prompt Cache Stuck at Tools-Only Floor

    Status: Open. Likely server-side regression. Filed pending OpenAI response.

    Reporter: OpenCode (third-party CLI/daemon built on top of @opencode-ai/codex-provider, a custom AI SDK V2 adapter that speaks the same Responses API wire format as codex-rs).

    Affected: Codex Responses API via ChatGPT-subscription auth. Multiple GPT-5.4 and GPT-5.5 sessions show the same floor. Replicates across fresh sessions and multiple working directories.

    Executive summary

    usage.input_tokens_details.cached_tokens consistently caps at 4608 across an entire codex session, regardless of:

    • Model (gpt-5.5 and gpt-5.4 both affected)
    • Working directory / repo
    • Session length (turns 2–200+ all return 4608)
    • WS chain delta mode (delta=true with previous_response_id) vs full re-send (delta=false)
    • Instructions content (driver-only ~640 tok, or full system blob ~16k tok — same floor)
    • Wire shape of bundle items (raw-string content vs ContentPart[] array — same floor)
    • prompt_cache_key value (composite vs pure ${threadId} — same floor)
    • Bundle insertion position in input[] (mid-chain at lastUserIdx vs head at index 0 — same floor)

    The 4608 figure ≈ instructions + tools schema token count for our setup. It looks like the prefix cache hit ends precisely where tools ends and the conversation chain begins.

    A historical session under the same OpenCode binary on 2026-05-10 22:48–23:56 showed cache_read growing 35,840 → 49,152 → 117,248 → 137,216 → 181,248 across consecutive turns under delta=true AND delta=false. Cache worked until something changed between then and our next sessions.

    Sessions starting 2026-05-11 all stuck at 4608. Same daemon binary lineage (we restarted the daemon several times for unrelated reasons), same @opencode-ai/codex-provider code path, same upstream codex-cli reference at f7e8ff8e50 (then 76845d716b after pull).

    Reproduction

    1. ChatGPT subscription account, OAuth credentials, codex Responses API endpoint via WebSocket transport (or HTTP fallback — same result)
    2. model: "gpt-5.5" (also reproduces on gpt-5.4)
    3. store: false, prompt_cache_key: "<sessionId>", previous_response_id: <chained>
    4. instructions: stable text (we tested driver-only ~2.5KB and monolithic ~65KB; both hit 4608)
    5. Send a turn, observe usage.input_tokens_details.cached_tokens = 0 (fresh)
    6. Send 2nd turn with previous_response_id set
    7. Observe cached_tokens = 4608 regardless of input_tokens (which grows naturally as conversation extends)
    8. Send 100+ more turns. cached_tokens stays at 4608. Occasional one-turn jumps to 50,688 / 71,168 right after a chain reset, then back to 4608.

    Diagnostic data

    Sample USAGE telemetry across 49 turns of a single session

    [CODEX-WS] USAGE input_tokens=18512 cached_tokens=0     hasPrevResp=false  ← turn 1
[CODEX-WS] USAGE input_tokens=22098 cached_tokens=4608  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=22988 cached_tokens=4608  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=31345 cached_tokens=4608  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=42290 cached_tokens=4608  hasPrevResp=true
... 30+ more turns same pattern ...
[CODEX-WS] USAGE input_tokens=105448 cached_tokens=4608 hasPrevResp=false  ← chain reset
[CODEX-WS] USAGE input_tokens=97730  cached_tokens=50688 hasPrevResp=true   ← peak (1 turn)
[CODEX-WS] USAGE input_tokens=101093 cached_tokens=4608 hasPrevResp=false   ← back to floor

    Historical comparison (working session 2026-05-10)

    [CODEX-WS] USAGE input_tokens=36197  cached_tokens=0      hasPrevResp=false  ← turn 1
[CODEX-WS] USAGE input_tokens=36610  cached_tokens=35840  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=49562  cached_tokens=36352  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=50162  cached_tokens=49152  hasPrevResp=true
[CODEX-WS] USAGE input_tokens=53760  cached_tokens=53760  hasPrevResp=true   (effectively 100%)
... cache continues to grow as chain extends ...
[CODEX-WS] USAGE input_tokens=215840 cached_tokens=181248 hasPrevResp=false  ← 84% on full re-send

    Request body fingerprint (current broken sessions)

    {
  "model": "gpt-5.5",
  "instructions": "<driver text, ~640 tok, byte-stable across turns, sha256:24c40934fe30...>",
  "input": [
    { "role": "developer", "content": [{"type":"input_text","text":"<role_identity + opencode_protocol bundle, 3850 tok, byte-stable>"}] },
    { "role": "user",      "content": [{"type":"input_text","text":"<AGENTS.md + environment_context bundle, 3940 tok, byte-stable>"}] },
    /* ... conversation history reconstructed from previous_response_id chain ... */
  ],
  "tools": [/* 11 function tools, schema byte-stable */],
  "tool_choice": "auto",
  "parallel_tool_calls": true,
  "prompt_cache_key": "ses_1eca32cc1ffelyxaN1FlERPqRS",
  "store": false,
  "previous_response_id": "resp_0af46a9adff..."
}

    driverHash, developerBundle.totalChars, userBundle.totalChars, prompt_cache_key, model, tools — all verified byte-stable across turns via local telemetry. No drift.

    What we tried (none lifted the floor)

    1. ✅ Persona alignment: replaced our 27-line custom driver with upstream BaseInstructions::default() (275 lines from refs/codex/codex-rs/protocol/src/prompts/base_instructions/default.md, md5 7a62de0a7552d52b455f48d9a1e96016). No change.

    2. ✅ prompt_cache_key = thread_id (matching upstream client.rs:713): switched our adapter from a composite key to pure threadId. Verified the value is byte-stable across all turns. No change.

    3. ✅ Wire structure aligned to upstream: instructions = driver only; input[] opens with one developer bundle item + one user bundle item, mirroring build_initial_context() in core/src/session/mod.rs:2553-2761. No change.

    4. ✅ Bundle position at input[0]: confirmed bundles always at chain head, not mid-conversation (transport delta-slice correctly trims them out as already-in-chain). No change.

    5. ✅ Bundle content as raw string vs ContentPart[]: tested both shapes. Raw string matched pre-break (May 9) wire that historically cached well. Both forms hit 4608. Reverted to ContentPart[] for upstream alignment.

    6. ✅ finishReason fallback fix: when terminal response.completed event fails to arrive but text was emitted, we now default to "stop" instead of "unknown". Stops a runloop empty-round guard from issuing chain resets every few turns. Visibly stabilized chain (see First issue! Just want to say thank you! #6 in upstream impact below) but did not improve cache_read floor.

    7. ✅ Daemon restart: clean process state. Same floor.

    8. ✅ New session: cache=0 on turn 1 (correct), cache=4608 on turn 2 (stuck), confirmed across 4 fresh sessions.

    9. ✅ gpt-5.5 → gpt-5.4: same floor.

    10. ✅ Re-auth / new credentials: re-authenticated the ChatGPT subscription credential and started a fresh session — same floor on the new credential.

    Hypothesis status

    • Wire-shape mismatch (raw string vs ContentPart[], bundle position, instructions size): Falsified by experiments 4–5 above.
    • Account / credential state: Plausible — the historical working session used a credential that has since gone into 5h cooldown, so we cannot directly retest with it.
    • Server-side regression: Strongly supported by three open OpenAI issues:
      • openai/codex#20301 — "Low cache hit rate when Codex integrates with GPT-5.5"
      • openai/codex#21756 — "Conversation cache unexpectedly drops to nearly zero during short continuous sessions" (filed 2026-05-08, exact match for our symptom)
      • openai/codex#21796 — Codex's own engineers observe ~55% cache hit on byte-identical prefixes, indicating a broader cache anomaly even under ideal conditions

    All three issues open, no assignee, no resolution.

    Operational impact

    A single chat turn under cache=4608 with input ≈ 80k tokens costs full price for ~75k uncached tokens. The 5h ChatGPT subscription quota gets exhausted in roughly 10 minutes of active multi-turn use, where the same workload before the regression was sustainable for hours.

    For any third-party tool relying on the Codex Responses API for non-trivial multi-turn work — agent loops, repo-wide code edits, long debug sessions — the cache regression is a hard blocker.

    What we would like from OpenAI

    1. Acknowledge the regression in one of the three open issues.
    2. Confirm or refute server-side cache changes between 2026-05-10 and 2026-05-11.
    3. Document the actual prefix cache prefix calculation for Responses API requests with previous_response_id set — the public docs do not address chain-mode interaction with prefix cache, leaving a black-box where third-party clients cannot self-debug.
    4. Surface an extended_cache_signature or similar explicit field clients can use to confirm what prefix the server hashed for cache lookup. Right now we have no way to tell whether our prompts are byte-stable from the server's perspective.

    Repro environment

    • OpenCode daemon (TypeScript/Bun), @opencode-ai/codex-provider ^x.x
    • refs/codex submodule pinned at 76845d716b (rust-v0.0.2504301132-6092)
    • Linux x64, WSL2 Ubuntu
    • ChatGPT subscription auth (OAuth flow)
    • Tested: gpt-5.5, gpt-5.4
    • WebSocket transport (also reproduces on HTTP SSE fallback)
    • store: false
    • service_tier: undefined (default)
    • reasoning.effort: configured (no impact tested)

    Related references

    • Upstream wire layout: refs/codex/codex-rs/core/src/session/mod.rs:2553-2761 (build_initial_context())
    • Upstream cache key: refs/codex/codex-rs/core/src/client.rs:713 (prompt_cache_key = self.state.thread_id.to_string())
    • Upstream BaseInstructions default: refs/codex/codex-rs/protocol/src/prompts/base_instructions/default.md
    • OpenAI prompt-caching guide: https://developers.openai.com/api/docs/guides/prompt-caching

  18. MrLightspeed commented on May 16, 2026

    @MrLightspeed
    MrLightspeed
    on May 16, 2026

    019e2e4d-e84b-78e3-bfb1-0b0c923200d5

  19. 2799662352
    added a commit that references this issue on May 18, 2026

    chore(release): v4.3.0 - codex multi-provider (apiyi / right.codes / …

    c2ddec4

  20. danielsan1 commented on May 19, 2026

    @danielsan1
    danielsan1
    on May 19, 2026

    This might be connected openclaw/openclaw#84110 Started with 2026.5.12 update

  21. makoto-soracom commented on May 22, 2026

    @makoto-soracom
    makoto-soracom
    on May 22, 2026

    019e4e09-2723-74f0-ad76-496e30ab9a70

  22. makoto-soracom commented on May 22, 2026

    @makoto-soracom
    makoto-soracom
    on May 22, 2026

    I am not attaching the raw payload JSON publicly because it contains full instructions, tool schemas, tool outputs, local paths, and request metadata. I checked the relevant payloads locally and did not find API keys,
    bearer tokens, cookies, refresh tokens, or access tokens, but the raw files still contain more context than should be posted in a public issue.

    The interesting part is a sequence of three consecutive Responses requests in the same thread:

    • previous request
      • started: 2026-05-22T04:56:09.758Z
      • completed: 2026-05-22T04:56:16.306Z
      • input_tokens=29658
      • cached_input_tokens=26496
    • low-cache request
      • started: 2026-05-22T04:56:18.856Z
      • completed: 2026-05-22T04:56:21.451Z
      • input_tokens=30168
      • cached_input_tokens=3456
    • following request
      • started: 2026-05-22T04:56:23.098Z
      • completed: 2026-05-22T04:56:25.136Z
      • input_tokens=30394
      • cached_input_tokens=30080

    The middle request is the low-cache one. Comparing it with the previous request:

    • instructions are identical
    • tools are identical
    • model is the same: gpt-5.5
    • prompt_cache_key is the same
    • Codex WebSocket diagnostics report outcome=incremental
    • request_without_input_hash is stable
    • request_tools_hash is stable

    Timing also makes cache expiry unlikely:

    • previous request start -> low-cache request start: about 9.1s
    • previous response complete -> low-cache request start: about 2.6s

    So this looks like a single intermittent prompt-cache collapse inside an otherwise healthy same-thread incremental Responses API chain. The following request recovered immediately to cached_input_tokens=30080.

  23. kyle7zhang commented on May 29, 2026

    @kyle7zhang
    kyle7zhang
    on May 29, 2026

    I’ve encountered the same issue before.

    It seems to happen when the WebSocket of the backend-api/codex/responses reconnects. If the user continues speaking after being idle for a while, the connection reconnects, and the cached_tokens counter resets and starts over. I suspect this is related to the new retention/eviction policy in GPT-5.5, and possibly to the request no longer being routed to the same backend instance after reconnection (just an assumption).

    Image

    It seems much more noticeable when using Azure Codex, since we typically use HTTPS/SSE instead of WebSockets.

    For example,

    • Previous WebSocket request: input_tokens 183426, cached_tokens 152448 (expected)

      • id:"resp_0442f7dc3e2b541c016a1a095214588199bbeabf8cf52eef1e"
      • prompt_cache_key:"019e74ff-6cf1-7d40-80ce-0c8baa3ad6cf"

    • New WebSocket request: input_tokens 184739, cached_tokens 91520 (dropped a lot)

      • id:"resp_0fd6c96548b5311c016a1a099e911081938f04ba42de133e60"
      • prompt_cache_key:"019e74ff-6cf1-7d40-80ce-0c8baa3ad6cf"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingrate-limitsIssues related to rate limits, quotas, and token usage reporting

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions