Repository navigation
[Code Mode] Reproducible token amplification: one AGENTS.md rule cut cumulative input by 81.6% #41450
Description
[Code Mode] Reproducible token amplification: one AGENTS.md rule cut cumulative input by 81.6%
Read this first: immediate zero-patch mitigation
If Codex is performing many independent, read-only checks, add the following to
the project AGENTS.md, or use it as the first instruction in a new session:
In Code Mode, when the current-stage checks are already known to be independent,
read-only, and conflict-free, put them in one exec and run them concurrently.
Use Promise.allSettled when useful partial results should be preserved and inspect
every result; use Promise.all only when any failure must abort the batch. Keep
dependent, writing, approval-sensitive, adaptive, and waiting operations serial.
Do not poll a running process every second; use a wait window appropriate to its
expected duration. Do not repeat completed checks. These rules must not reduce
task scope, reasoning depth, verification, tool coverage, or answer quality.
This is a scheduling instruction, not a model downgrade or a task shortcut.
In my controlled A/B test, it reduced cumulative logical input by 81.606%
while preserving 100% task coverage:
| Control: serial tool plan | Treatment: scheduling guidance | |
|---|---|---|
| outer model cycles | 60 | 15 |
| cumulative logical input | 11,100,973 | 2,041,915 |
| cumulative cached input | 9,980,996 | 1,827,703 |
| required checks | 50/50 | 50/50 |
| nested tool calls | 86 | 86 |
| errors | 0 | 0 |
That is 9,059,058 fewer logical input tokens, approximately 82% less
for this dense high-context workload. The same task was completed, the same
nested calls were made, and no model, tool, reasoning, context, compaction, or
verification capability was reduced.
The complete investigation, including the local-history analysis, deterministic
reproduction, wait/retry/fallback tests, context sweep, limitations, and source
reference is in this issue. Earlier related Code Mode evidence is in
#35050; the related large-input
quota symptom is in #41369.
What is the bug/failure mode?
The problem is not that GPT-5.6 Sol is abnormal on every turn.
The measurable failure mode is:
cumulative logical input ≈ Σ input presented to each outer model cycle
≈ carried context × outer model cycle count
For already-known independent checks, an inefficient plan can repeatedly return
to the outer model after one small tool call:
model → tool A → model → tool B → model → tool C
The growing transcript is then logically presented again and again. A high
cached-input ratio is expected in this pattern; it indicates repeated,
cacheable history, not by itself a cache or wire-transport defect.
The lower-cost equivalent for known-independent work is:
model → one exec(Promise.all/Promise.allSettled(A, B, C)) → model
This does not apply to dependent, dynamic, writing, side-effectful,
approval-gated, or waiting operations. Those should remain serial.
The issue is therefore a client/model-interaction scheduling failure that can
amplify quota usage. It may explain a substantial part of the perceived
“quota shrink” symptom in dense tool workloads, but it is not a claim that every
quota report has the same cause or that every billing issue is explained here.
Why can the same Sol look normal sometimes?
A streamed scan of recent local session metadata found:
- 60 session files;
- 205 identifiable Sol-labelled turns;
- 144 main turns and 61 child turns; and
- normal, moderate, and extreme outliers.
The observable difference was the outer-cycle shape, not merely the model name.
Normal turns tend to batch independent calls. Abnormal turns tend to split them
into singleton calls and re-enter the model repeatedly while context grows.
Examples from same-session state changes were:
| turn shape | input delta | outer cycles | exec / nested calls |
waits | compactions |
|---|---|---|---|---|---|
| short/normal | 21,667 | 1 | 0 / 0 | 0 | 0 |
| abnormal | 4,528,553 | 51 | 33 / 35 | 5 | 0 |
| extreme | 36,040,574 | 303 | 245 / 272 | 49 | 2 |
| later shorter turn | 621,343 | 3 | 1 / 3 | 0 | 0 |
Across 110 anchored main Sol turns, the strongest predictor was average
per-cycle input multiplied by outer cycles:
- Pearson correlation:
0.9905; - Spearman correlation:
0.9841.
The corrected linear fit was:
inputDelta = -34,031 + 0.9230 × (average_last_input × outer_cycles)
R² = 0.9811
Singleton-exec ratio alone was not a sufficient historical classifier because
the metadata cannot prove whether every singleton was semantically independent.
The deterministic fixture below isolates the causal cost of serializing work
that is known to be independent.
Reproduction with the unmodified official client
The primary A/B used the installed, unmodified official Codex binary:
codex-cli 0.150.0-alpha.12.2;- binary SHA256:
34E9CFE7D5BBCEC306FE6AB3FD502A713A7A1F0FB644C11AD2990FC80599FD4F; - temporary isolated
CODEX_HOME; - deterministic localhost fake Responses server;
- 180K synthetic context;
- 50 independent read-only checks; and
- the same tool workload and coverage requirements in both arms.
The client was configured for GPT-5.6 Sol. The fake server replaced only the
upstream at the network boundary, recorded usage/tool/completion events, and
never called a real model, production OpenAI endpoint, or production Sub2API.
Control and treatment used the same official client path. The only experimental
variable was the scheduling guidance that biases known-independent calls toward
one programmatic batch.
The treatment was repeated:
| run | outer cycles | logical input | cached input | exec / nested |
coverage |
|---|---|---|---|---|---|
| control | 60 | 11,100,973 | 9,980,996 | 59 / 86 | 50/50 |
| treatment 1 | 15 | 2,041,915 | 1,827,703 | 14 / 86 | 50/50 |
| treatment 2 | 15 | 2,041,940 | 1,827,727 | 14 / 86 | 50/50 |
Both treatment runs had zero errors and complete coverage. A mixed-command
fixture was also repeated: both arms executed the same command shape
(Get-Content×49, Select-String×7, Get-Item×6) and covered all 20 checks;
only the outer scheduling shape changed.
This fake-server test is intentionally not described as reproducing Sol's
learned sampling policy. It tests the real official client path and makes the
cost of the two scheduling shapes deterministic. The local session history
supplies the real normal/abnormal variation; the isolated fixture supplies
repeatable causal accounting.
Context-size sweep
A separate deterministic harness sweep used the same 20 independent checks at
different synthetic context sizes. Its logical usage is approximately JSON
characters divided by four, so these values demonstrate structural amplification
rather than production tokenizer or billing values.
| context | batched input | serial input | reduction | coverage |
|---|---|---|---|---|
| 20K | 97,999 | 735,875 | 86.7% | 20/20 both |
| 100K | 491,240 | 2,693,946 | 81.8% | 20/20 both |
| 180K | 1,235,604 | 5,004,702 | 75.3% | 20/20 both |
| 220K | 1,731,486 | 6,283,941 | 72.4% | 20/20 both |
At 180K with 50 checks, the serial plan reached 11,099,443 input versus
2,038,803 for the batched plan, with 50/50 coverage in both arms.
The headline range is therefore a measured high-context amplification range,
not a universal promise for every task.
Accounting for the originally quoted 11.16M turn
One reproducibility correction is important: the local turn previously described
as an 11.16M GPT-5.6 Sol turn is identified by its recorded metadata as
gpt-5.6-luna, not Sol.
That turn contained:
- 95 usage cycles;
+11,156,761input;+10,507,264cached input;- 86
execinvocations; - 87 nested calls;
- 85 singleton exec cells and one two-call exec cell;
- wait polling: 0;
- retry/fallback/incomplete markers: 0; and
- approximate retained tool output of only 3.439% of the input delta.
The nearby correctly identified Sol turn had 50 cycles and +4,416,127 input,
with 38 singleton nested-call execs, wait 0, and no retry/fallback/hidden-model
signal.
The numerical phenomenon is real, but the original model label must not be
repeated. The available evidence does not establish that Sol is intrinsically
more susceptible than Terra or Luna.
The JSONL schema does not provide mutually exclusive labels for “normal tool
continuation” and “inefficient serialization”. I therefore do not invent an
exact semantic partition. The logs prove that the dominant identifiable shape
was 85 singleton exec cells plus one two-call exec, while wait, retry, fallback,
and hidden-agent signals were absent.
Other hypotheses tested
Wait polling
A deterministic approximately-30-second process was run with two wait policies:
| policy | cycles | logical/cached input | short waits |
|---|---|---|---|
| one-second polling | 7 | 79,550 / 61,753 | 6 |
| reasonable wait window | 2 | 22,046 / 9,997 | 0 |
One-second polling is a real secondary amplifier:
wait → model → wait → model
It cannot explain the largest target ledgers by itself, and the 11.16M target
turn had no wait events.
Retry and fallback
One pre-output connection failure added approximately one cycle in the fake
retry test. A fake WebSocket failure followed by retry and HTTP fallback
completed with the same coverage and essentially the HTTP-baseline logical
input. These mechanisms can amplify usage, but they are not the primary cause
of the measured serial-tool explosion.
Tool output
Retained tool output was approximately:
- 10.13% of the input delta in the nearby real Sol turn;
- 3.439% of the input delta in the 11.16M Luna turn.
Repeated context, not newly produced tool output, was the main contributor.
Provider and hidden agents
The local target provider metadata was openai, and the official client
reproduced the structure against a local fake Responses endpoint. Sub2API is
therefore not required to reproduce this particular behavior.
No hidden Luna/Terra/guardian/auto-review/subagent signal was present in the
target attribution ledger. A fair controlled Sol-versus-Terra-versus-Luna
susceptibility comparison has not been established.
Why the mitigation preserves capability
The A/B test kept the following unchanged:
- official binary;
- model and model configuration;
- context;
- tool workload;
- 50 independent checks;
- 86 nested calls;
- verification requirements;
- final coverage; and
- error handling.
The instruction only changes this representation:
model → tool A → model → tool B → model → tool C
into this representation when the calls are already known to be independent:
model → one exec(Promise.all/Promise.allSettled(A, B, C)) → model
All results must still be preserved and inspected. Promise.allSettled is used
when partial failures should not cancel useful results. Promise.all is used
only when any failure must abort the batch.
Dependent, state-changing, writing, approval-sensitive, adaptive, and waiting
operations remain serial.
This is scheduling efficiency, not reduced reasoning, reduced verification, or
reduced tool capability.
Candidate upstream reference change
A minimal reviewable source-level reference is available here:
It adds safe scheduling boundaries to the Code Mode exec description and adds
a regression test. It does not modify the runtime binary used for the A/B test.
No Responses serializer, model metadata, context setting, compaction setting,
reasoning setting, tool implementation, retry logic, transport, or binary is
changed.
This fork commit is a maintainer-facing reference, not a request for users to
install a patched binary or maintain a private Codex fork. The practical
zero-patch mitigation is the supported first-message or AGENTS.md instruction.
If upstream later fixes the default scheduler, the workaround should be removed
after verification, and this fork reference should no longer be needed.
Proposed acceptance criteria for an upstream fix
An upstream implementation should be tested against the same fixture and should:
- reduce outer cycles and cumulative logical input in the serial failure case;
- preserve 100% of required checks, nested calls, error handling, and final
findings; - leave context, compaction, reasoning, model, tool, retry, and transport
capabilities unchanged; and - make the AGENTS.md workaround removable after an official release is verified.
Summary
GPT-5.6 Sol is not abnormal on every turn.
Dense tool workflows can sometimes enter a high-outer-cycle scheduling state.
When a large context is replayed across many unnecessary model cycles, cumulative
logical input grows into the millions.
The controlled result was:
BEFORE: 60 cycles, 11,100,973 input, 9,980,996 cached
AFTER: 15 cycles, 2,041,915 input, 1,827,703 cached
Both arms completed the same required work with 100% coverage and zero errors.
This is a reproducible Code Mode token-amplification failure mode that may
explain a significant portion of the perceived “quota shrink” effect in dense
tool workloads. It is actionable today through an AGENTS.md scheduling rule,
while the source-level reference provides a concrete upstream regression test.
Activity
github-actions commentedon Aug 29, 2026
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
larryllr commentedon Aug 29, 2026
Validation update
The focused Rust validation now passes on the exact fork commit linked above:
This ran in an isolated WSL environment with an offline dependency cache and a
local ext4 target. The new
code_mode_only_description_includes_scheduling_boundariestest passed along with the existing 35 tests.
rustfmt --checkalso passes.The native Windows workspace test was separately attempted but cannot link on
this host because its Visual Studio installation has no Windows SDK. That is a
host toolchain limitation, not a source diagnostic; the Linux focused compile
and test path completed successfully. No patched binary was built or used for
the A/B numbers.
[-][Code Mode] Dense tool-call turns can waste 72-82% of cumulative input while preserving the same work[/-][+][Code Mode] Measured zero-capability-loss mitigation for dense-tool cumulative input amplification (72-82%)[/+]larryllr commentedon Aug 29, 2026
Scope clarification: keeping this as an independent mitigation issue
I am keeping this issue open because its purpose is narrower and actionable than
the neighboring symptom reports:
quota depletion and large cached-context replay in an ordinary task;
serialization of independent calls; and
zero-capability-loss mitigation: the same unmodified client, context, task,
tools, nested-call count, and 100% coverage, with cumulative input reduced
from
11,100,973to2,041,915(81.606%).The mechanism is related, but this issue is not another unquantified report of
high usage. It supplies the reproducible intervention, the remaining measured
input, the context sweep (
72.4%–81.8%reduction in high-context cases), theretry/wait/fallback exclusions, the supported
AGENTS.md/first-messageworkaround, and a reviewable fork commit. Those are the acceptance artifacts
needed to turn the reports into a fix.
If maintainers ultimately require one tracker, please cross-link or consolidate
without dropping this issue's A/B tables and mitigation evidence. Until then,
keeping it open makes the solution discoverable as a standalone actionable
report rather than burying it in a symptom-only thread.
[-][Code Mode] Measured zero-capability-loss mitigation for dense-tool cumulative input amplification (72-82%)[/-][+][Code Mode] Reproducible token amplification: one AGENTS.md rule cut cumulative input by 81.6%[/+]larryllr commentedon Sep 5, 2026
September 5 update: current official client improves batching, but the amplification path remains
I retested after updating to the unmodified official
codex-cli 0.153.4. This is an evidence update, not a claim that a matched before/after production benchmark has been completed.1. A real forked thread shows the scheduling instruction can take effect
A forked local thread with the supported scheduling instruction loaded completed its first Luna turn with:
execcellsexeccellsexeccellsPromise.allSettledbatchesThe first four batches had sizes
4, 4, 4, 3; this is the intended change in outer-cycle shape. The fork still inherited a large context, so its first turn was not a fair zero-context cost comparison and is not used to claim another percentage saving.2. Astra has explicit batching behavior, but it is not enforced
The same fork was then switched to
gpt-6-astra. Four anchored Astra turns recorded:execcells / nested callsexeccellsexeccellsPromise.allSettledbatchesThe largest Astra turn was
2,671,170input across 19 cycles. Later turns were593,256/9 cycles,213,098/3 cycles, and363,244/5 cycles. Thus Astra is materially better than the historical fully-serial traces, but 52% of itsexeccells in this sample were still single-call cells. This is a partial mitigation, not a reliable client-side fix.This is consistent with the merged Astra catalog addition in #42607: Astra has model-specific wording to batch independent reads/searches with
Promise.allSettled, while retaining serial behavior for dependent, mutating, approval-sensitive, or waiting work. Model-facing guidance can improve behavior, but it cannot guarantee that every eligible action is batched.3. Production relay telemetry confirms the same shape is not Sol-only
A read-only seven-day aggregation from one self-hosted relay's
usage_logsprimary database recorded repeated large cached-context requests across multiple models. The relay is used here as an upstream usage ledger only; it does not establish causation.input + cache_read)The two largest Sol-correlated sessions had 523 rows / 58.81M logical input and 314 rows / 51.65M logical input. These values do not prove that every row is an independent Codex outer cycle; they do demonstrate the repeated-large-context usage shape in live traffic. The database contains no prompt text in this report.
Bottom line
The supported
AGENTS.md/ first-message scheduling rule remains the immediately usable, zero-patch mitigation for known independent read-only work. It preserved required coverage in the controlled A/B in this issue and improves real scheduling behavior in the new fork.However, the underlying risk remains open:
An upstream durable fix should put the scheduling boundary into the shipped Code Mode prompt contract (or enforce safe batching in the client where semantics permit), and separately eliminate no-progress wait/poll model re-entry. Until an official release passes the same matched fixture, users should treat the instruction as a mitigation, not as proof that the bug is fixed.
Related: #35050, #32503, #35618, #37299, #38495, #40897, #41369, and #42607.
larryllr commentedon Sep 5, 2026
Proposed durable upstream fix: ship the scheduling contract in the Code Mode
execschemaThe current
AGENTS.mdrule is useful immediately, but it cannot repair a persisted conversation: the official session loader preferentially restoressession_meta.base_instructionsfrom history. That is why editing AGENTS.md today only reliably affects a new session/fork.The smallest client-owned place that does reach an old thread's next turn is the Code Mode
exectool description. Each sampling request usesstep_context.tool_router.model_visible_specs()(core/src/session/turn.rs), and the tool router is captured again for that step. Code Mode decorates the nested-tool specs viabuild_exec_tool_descriptionincode-mode-protocol/src/description.rs. Tool schemas are sent with the Responses request, independently of the historicbase_instructionsvalue.I rebased a one-file candidate on current upstream
main(588b781ab4924ce7352488394028e63d74cf807f):larryllr@56daa6a
It adds this only for
code_mode_only:The diff also adds a focused unit test that verifies the contract is present in Code Mode-only descriptions and absent from direct-tool descriptions.
Why this is preferable to automatic client parallelization
Arbitrary Code Mode JavaScript can have hidden data dependencies and side effects. The client cannot safely reinterpret a model-written serial program as parallel. This proposal preserves that semantic boundary: only the model classifies calls as eligible; the runtime supplies the missing, stable contract for every supported model.
It does not alter model selection, reasoning, context window, compaction, enabled tools, MCP, tool output limits, or task/check coverage. It therefore also works for a resumed high-context thread without rewriting or discarding that history.
Controlled evidence and acceptance criterion
The strict official-0.153.4 localhost-fake A/B already recorded in this issue used a clean home, no AGENTS files, ~180K retained context, and 50 independent read-only checks. With the same unmodified official client:
The candidate should be accepted only with a maintained fixture that (1) performs the same complete checks, (2) exercises both fresh and resumed large-context histories, (3) asserts the generated
execschema contains the contract for every Code Mode model, and (4) shows fewer outer cycles without reduced coverage. The current user-side evidence is strong for the mechanism and mitigation, but a maintainer-run deterministic fixture is needed before claiming the official release fixes it.PR status
The branch is pushed and reviewable at the commit above. I attempted to open
openai/codexPRlarryllr:codex-mode-scheduling-guidance -> main; GitHub rejected the API call withCreatePullRequestpermission despite the fork being public and the account having admin/repo scope. No PR exists yet, so there is no false PR claim here. A maintainer or a GitHub UI flow that permits external PR creation can use the commit directly.