Every Code client: elevated cache misses

What version of Codex is running?

0.47.0

What subscription do you have?

Pro

Which model were you using?

gpt-5-codex

What platform is your computer?

Darwin 25.0.0 arm64 arm

What issue are you seeing?

I couldn't figure out why I was blowing through a week's worth of usage limits in a single day.

After digging into the response data it turns out that cached_tokens are ALL over the place when using a ChatGPT login at the moment.

On historical data I was seeing around a 20:1 cached to input ratio. I'm now seeing a ~1:1 cached to input ratio over the last 24-48 hours. This is hugely impacting usage limits AND performance. I tried switching over to using an API key with codex and I was back to seeing the usual 20:1 ratio on average.

What steps can reproduce the bug?

Log the usage from responses. Compare ChatGPT account usage vs API key usage. Notice that the cached_tokens increases normally for API keys - as requests grow in size with no changes, so does the cache. This is not the case for ChatGPT logins - caches are staying fixed at the same value and sometimes even decreasing during the conversation!

What is the expected behavior?

As the input tokens increase, the cached tokens should increase as well.

Additional information

I built a version of codex with the response payloads logged to detect this.

I'm logged in via ChatGPT and can corroborate that the token usage seems to have increased massively: my weekly limit has completely disappeared running gpt-5-codex MEDIUM for some very basic work over a couple of hours.

Yup, that experience is was what lead me to find this. I don't know how OpenAI's usage limits work, but it looks like it's budget based. Given that cached tokens are 1/10 of the price of normal tokens this hugely increases usage. At least 5x and maybe up to 200x in some situations if my math is right.

@zemaj

Thanks for flagging - can you provide some session ids to help us investigate?

@zemaj can you reproduce this on the Codex CLI specifically? Prompt cache can be invalidated by many things, e.g. changes in tool definitions. I'd prefer to isolate this investigation to code that we manage, i.e. openai/codex specifically.

@rickydutton-businessfitness can you provide session ids from /status or codex resume from instances where usage seemed significantly higher than you'd expect?

I felt like the same was going on this past week too. For my first time I hit my PRO limit :( Boring 48 hours here..

Edit: I guess I'm not really seeing this, just had some large repo inputs (repeated over and over sadly) that ate mine up. This is my last set of sessions' final stats. :(

{
"input_tokens": 5248168,
"cached_input_tokens": 5005184,
"percent_cached": 95.37011772488991
}
{
"input_tokens": 3234910,
"cached_input_tokens": 3104384,
"percent_cached": 95.96508094506493
}
{
"input_tokens": 1724316,
"cached_input_tokens": 1625856,
"percent_cached": 94.28990973812225
}
{
"input_tokens": 28639571,
"cached_input_tokens": 28091648,
"percent_cached": 98.08683237608552
}
{
"input_tokens": 9910693,
"cached_input_tokens": 9665792,
"percent_cached": 97.52892153959365
}
{
"input_tokens": 52580,
"cached_input_tokens": 41728,
"percent_cached": 79.3609737542792
}
{
"input_tokens": 8946727,
"cached_input_tokens": 8584576,
"percent_cached": 95.9521398160467
}
{
"input_tokens": 39360050,
"cached_input_tokens": 38575744,
"percent_cached": 98.00735517358336
}
{
"input_tokens": 13327109,
"cached_input_tokens": 12957184,
"percent_cached": 97.22426671831077
}
{
"input_tokens": 3380166,
"cached_input_tokens": 3268480,
"percent_cached": 96.6958427485514
}
{
"input_tokens": 6260352,
"cached_input_tokens": 5866240,
"percent_cached": 93.70463513872703
}
{
"input_tokens": 30300543,
"cached_input_tokens": 29443456,
"percent_cached": 97.17138072410123
}
{
"input_tokens": 1975752,
"cached_input_tokens": 1809792,
"percent_cached": 91.60016034401079
}
{
"input_tokens": 169350,
"cached_input_tokens": 148480,
"percent_cached": 87.67640980218484
}
{
"input_tokens": 3708602,
"cached_input_tokens": 3556480,
"percent_cached": 95.89813088597806
}
{
"input_tokens": 19184139,
"cached_input_tokens": 18741504,
"percent_cached": 97.69270333164287
}
{
"input_tokens": 334078,
"cached_input_tokens": 315136,
"percent_cached": 94.33006663114601
}
{
"input_tokens": 10075678,
"cached_input_tokens": 9666176,
"percent_cached": 95.93573752555412
}
{
"input_tokens": 238029,
"cached_input_tokens": 223488,
"percent_cached": 93.89108049859472
}
{
"input_tokens": 358765,
"cached_input_tokens": 333184,
"percent_cached": 92.86970579627332
}
{
"input_tokens": 19411406,
"cached_input_tokens": 18838016,
"percent_cached": 97.04611814311647
}
{
"input_tokens": 8237601,
"cached_input_tokens": 7972096,
"percent_cached": 96.77691356986094
}
{
"input_tokens": 10914466,
"cached_input_tokens": 10685568,
"percent_cached": 97.90280165790979
}
{
"input_tokens": 120433,
"cached_input_tokens": 96128,
"percent_cached": 79.81865435553378
}
{
"input_tokens": 944886,
"cached_input_tokens": 871424,
"percent_cached": 92.2253054865878
}
{
"input_tokens": 41656587,
"cached_input_tokens": 40803072,
"percent_cached": 97.95106833884398
}
{
"input_tokens": 4769765,
"cached_input_tokens": 4615296,
"percent_cached": 96.76149663557848
}
{
"input_tokens": 9326943,
"cached_input_tokens": 9018880,
"percent_cached": 96.69706355019002
}

@dylan-hurd-oai

Also I went back over my numbers, while watching a certain other LLM CLI (starts with G...) on its ~12th hour of walking in circles to solve a Gradle issue, and it looks like I got hit with a huge unusual usage % dent suddenly.

This only covers local codex CLI, no cloud usage included:

Period (UTC) | Token pattern | Secondary % pattern | Interpretation
Oct 18 03:54 → Oct 20 00:00 | ~0.5–0.8 B tokens/h | 0 → 24 % | Normal proportional rise
Oct 20 00:00 → Oct 21 03:00 | Throughput doubles (~1.4 B tokens/h) | 24 → 29 % | Still proportional
Oct 21 03:04 | ~0.55 B tokens | +46 % jump (29 → 75 %) | Accounting flush; token rise too small     <----------
Oct 21 → Oct 23 02:35 | ~1.0 B tokens total | 75 → 100 % | Continued accumulation at inflated rate

I guess that answers the question I saw on reddit or elsewhere if the Oct 20th date was going to end our fun with the cloud Codex version :(

I am curious, might try to compute it in the near future, if the token rates for cloud vs local are comparable. Sadly I have found the model creating very-verbose tests and doing stupid things a LOT lately, such as running a giant combined set of tests, not finding the failures in the truncation-surviving context, then either dice roll reading tons of log context to find issues OR going down bad paths missing errors/warnings/context (which often turns it into a loop or worse) via both, but the cloud one seems to "give up" a lot more even after just 15 minutes due to some tests taking too long or it doing it's stupid "I will wait 60 seconds" <waits 5 seconds> behaviors.

I would plead to @dylan-hurd-oai or other OAI folks to consider some kind of scheme to where model failures, loops, mistakes, etc do not go against our quotas. With most other API based usage devs are more in-control of the total session/conversation's token payloads, flows, and so forth. But with the nature of codex we do not have much control over the iterations in a single turn. A single request to Codex can end up with millions of tokens we didn't ask for, didn't want, and were from bad feedback loops, mistakes, bugs, timeouts, other issues more unique to Codex that we cannot easily prevent in the blink of an eye. And they snowball. The fact that I can give an exact task to Codex Cloud and it fails/gives up after 15mins and how many millions of tokens into the toilet, but then start an identical session from the same cached overlay with the same repo and same prompt, and it nails it, is one of my bigger personal frustrations lately. I felt like I was at fault with my processes, prompts, etc but I clearly see a lot of 'random' coming into play that feels like "which model did I get this random spin, is it one that will try a few things, find failing tests, and give up or one that will use the tests to debug or at least document some progress or do ANYTHING useful with all those tokens and resources consumed???".

I will say, that other LLM is MUCH MUCH worse in every respect of that, but I think we'll find ourselves questioning our token usages a lot more heavily as of this week. :(

/rant

Ok after going into a deeeeeep dive on this, I found the root cause. Me.

The latest version of codex uses

conversation_id
session_id

in the headers and the fork I was testing on did not use these. Despite the prompt_cache_key being present in the body of the request it was being invalidated and changed to a different key in the response. This meant that basically every request had an almost fresh cache. Adding in these headers and matching the current request format resolved the problem.

I did see some strange behaviour in the live codex version (it looks like the cache gets partly invalidated on the first assistant response) but that's limited in impact and the same across the API and all other locations. This was what lead me to believe that this issue was present in 0.47 but after digging deeper I realised it was not the same issue.

The caching issue with the missing headers likely affects people using earlier version of codex (0.42 and below), but not the latest release.

So closing this out. Sorry!

I'm on various older franken-forks here too...Hmm guess I will have to dig deeper there too. Thanks for sharing!

@techjoec

@techjoec appreciate the detailed feedback re: Codex Cloud. Will share this with the team!

@zemaj

Hi @zemaj - reopening this thread because we're still seeing increased cache misses from the code_cli_rs relative to codex cli. Going to loop @tibo-openai in here as well.

Sure thing - if you need anything from my end let me know. Last time I did a comparison a few days ago latest release was comparable with codex, but hard to compare. Requests are all stable (i.e. no tool/history changes during session).

The codex team has done a deep investigation of all issues related to rate limits and token usage accounting, and we've implemented a number of adjustments and bug fixes. See here for a summary. This includes some changes related to cache miss accounting.

@etraut-openai

@etraut-openai It's been working great lately! Thank you so much!

Navigation Menu

Repository navigation

Every Code client: elevated cache misses #5556

What version of Codex is running?

What subscription do you have?

Which model were you using?

What platform is your computer?

What issue are you seeing?

What steps can reproduce the bug?

What is the expected behavior?

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Every Code client: elevated cache misses #5556

Description

What version of Codex is running?

What subscription do you have?

Which model were you using?

What platform is your computer?

What issue are you seeing?

What steps can reproduce the bug?

What is the expected behavior?

Additional information

Activity

rickydutton-businessfitness commented on Oct 23, 2025

zemaj commented on Oct 23, 2025

dylan-hurd-oai commented on Oct 23, 2025

techjoec commented on Oct 24, 2025

techjoec commented on Oct 24, 2025

zemaj commented on Oct 24, 2025

techjoec commented on Oct 24, 2025

dylan-hurd-oai commented on Oct 24, 2025

dylan-hurd-oai commented on Oct 31, 2025

zemaj commented on Nov 1, 2025

etraut-openai commented on Nov 26, 2025

zemaj commented on Nov 26, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions