GPT-5.6 all-turns reasoning is double-counted, causing auto-compaction with ~20% context remaining #39767

What version of Codex CLI is running?

codex-cli 0.148.0

What subscription do you have?

ChatGPT Pro

Which model were you using?

gpt-5.6-sol

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What terminal emulator and version are you using (if applicable)?

No response

Codex doctor report

What issue are you seeing?

Summary

In a long-running gpt-5.6-sol rollout whose initial session metadata records Codex CLI 0.144.5, Codex automatically compacted the conversation while approximately 20% of the effective context window was still available. The thread may have subsequently been resumed by newer Codex builds, but the rollout does not record a client version for each resume or turn.

Analysis of the rollout shows that the API-reported token usage already accounted for historical reasoning items. However, Codex behaved as if server_reasoning_included were false and added an estimated size for the same historical reasoning items again.

This caused historical reasoning to be double-counted for the auto-compaction decision.

Across 76 automatic compactions in the analyzed rollout:

• The current server_reasoning_included = false calculation crossed the auto-compaction limit in 76/76 cases.
• The server_reasoning_included = true calculation crossed the limit in 0/76 cases.

Analyzed rollout

• Model: gpt-5.6-sol
• OS: Windows
• Session creation version: Codex CLI 0.144.5
  • This is the version recorded in the rollout's initial session_meta.
  • The thread was long-running and may have been resumed by newer Codex builds.
  • The rollout does not record the client version for every resume or turn.
• Current source inspection: the same accounting path is still present on main at commit 3b45c29062ff0e76e71c91b6753290400e7fa8da (2026-08-19).
• Effective context window in the analyzed rollout: 272,000 tokens
• Auto-compaction limit: 244,800 tokens
• Reasoning context: all_turns
• Rollout ID: 019f74ae-8c3d-77e0-8725-2b2c99546fba

Therefore, this report should not be interpreted as being limited to CLI 0.144.5. That version is directly confirmed by rollout metadata; newer builds remain susceptible at the source-code level when the server already includes historical reasoning but the reasoning-included signal is absent or not propagated.

GPT-5.6 defaults to persisted all_turns reasoning according to the official model guidance:

https://developers.openai.com/api/docs/guides/latest-model

Actual behavior

Codex computes the auto-compaction token usage through:

last_tokens
    .saturating_add(self.get_non_last_reasoning_items_tokens())
    .saturating_add(items_after_last_model_generated_tokens)

when server_reasoning_included is false.

For this rollout, the historical reasoning represented by get_non_last_reasoning_items_tokens() was already reflected in the API-provided token usage, so this estimate counted the same reasoning a second time.

As a result, automatic compaction was triggered significantly before the real request context reached the configured limit.

Evidence

1. Reconstructing server input-token accounting

I reconstructed the model-visible request history across 1,755 consecutive cross-turn transitions and compared the next response's server-reported input_tokens against two predictions:

The server-reported input_tokens closely match the reconstruction that includes historical reasoning.

The alternative hypothesis—that the server excludes previous reasoning—has a median error larger by more than 13,000 tokens.

This indicates that historical reasoning was already included in the service token accounting for this rollout.

2. Replaying every automatic compaction

I replayed all 76 automatic compaction decisions using both branches of get_total_token_usage().

There were no preceding new_context_window tool calls or other manual compaction triggers. These were automatic token-limit compactions.

3. Concrete example

One compaction was triggered with:

last server-reported total       = 209,336
historical reasoning estimate    =  33,371
items after last model response  =   4,726
auto-compaction limit            = 244,800

Current calculation:

209,336 + 33,371 + 4,726 = 247,433
247,433 > 244,800

Therefore Codex compacts.

Calculation without double-counting historical reasoning:

209,336 + 4,726 = 214,062
214,062 < 244,800

With a 272,000-token effective context window, this leaves approximately:

(272,000 - 214,062) / 272,000 = 21.3%

This matches the observed UI state: auto-compaction occurs with roughly 20% context remaining.

Relevant code path

At the start of every regular turn, Codex resets the flag:

sess.set_server_reasoning_included(/*included*/ false).await;

The flag becomes true only when a ServerReasoningIncluded(true) event is received.

For SSE, this event is emitted only when the response contains the x-reasoning-included header.

For WebSocket transport, the same information is derived from the WebSocket handshake header and emitted for each request made over that connection.

The token calculation then uses the flag here:

pub(crate) fn get_total_token_usage(
    &self,
    server_reasoning_included: bool,
) -> i64

When the flag remains false, historical reasoning is estimated and added again.

Relevant files:

• codex-rs/core/src/tasks/regular.rs
• codex-rs/core/src/context_manager/history.rs
• codex-rs/codex-api/src/sse/responses.rs
• codex-rs/codex-api/src/endpoint/responses_websocket.rs

Likely root cause

For this session, the client behaved as if server_reasoning_included were false even though the returned token usage already accounted for historical reasoning.

This suggests one of the following:

1. The backend route did not emit x-reasoning-included even though its usage accounting included historical reasoning.
2. The header was emitted upstream but was not propagated through the relevant response or WebSocket path.
3. The server/client contract for GPT-5.6 all_turns token accounting is inconsistent across routes.

I do not have a raw packet capture of the response headers, so the evidence establishes that the client remained on the false accounting branch, but does not distinguish which layer failed to propagate the signal.

What steps can reproduce the bug?

1. Start or resume a long-running Codex CLI thread using gpt-5.6-sol with persisted reasoning.context = all_turns.
2. Continue the thread across many model sampling turns so that multiple historical encrypted reasoning items accumulate.
3. Continue until the context indicator shows approximately 20% remaining.
4. Submit another normal user turn.
5. Observe that automatic compaction can run even though the server-reported usage plus the newly appended local items remains below the configured auto-compaction limit.
6. Inspect the rollout's token_count, response_item.reasoning, and compacted records and replay get_total_token_usage() with server_reasoning_included set to both false and true.

This appears to depend on whether the reasoning-included signal is propagated by the selected backend and transport route. The analyzed rollout provides 76 occurrences of the behavior; the numerical replay is included in the issue description.

What is the expected behavior?

If the server-provided token usage already accounts for historical reasoning, Codex should not add get_non_last_reasoning_items_tokens() again.

The auto-compaction decision should use:

server-reported token usage
+ items added locally after the last model response

without re-estimating reasoning that the server has already counted. Automatic compaction should not run while the resulting usage remains below the configured threshold.

Additional information

No response