Skip to content

measured cost of [false] goal continuations across 3,808 sessions #44909

Open

Description

@ivanopcode

What version of Codex CLI is running?

rust-v0.154.0; rollouts from CLI 0.144–0.154; macOS

What subscription do you have?

ChatGPT Pro 20x

Which model were you using?

No response

What platform is your computer?

No response

What terminal emulator and version are you using (if applicable)?

No response

Codex doctor report

What issue are you seeing?

Summary

With a goal active, Codex restarts the model the moment the thread goes idle, including after turns that made no tool call. The continuation prompt accepts only a live poll as waiting, and the one tool that could wait without spending a turn, clock.sleep, is registered for a single model in the catalog. Waiting for an external process becomes a spin loop where every iteration rereads the full context. On my rollouts, 1.4% of sessions with a goal used 49% of all input tokens. Related reports: #28923, #40929, #24531.

Mechanism

  • extension.rs:175 on_thread_idle calls continue_if_idle, which calls start_turn_if_idle at runtime.rs:454 with no delay. Measured gap between the end of one turn and the next continuation: 0.02–0.05 s, median 0.03 s.
  • token_budget defaults to None (runtime.rs:348).
  • The goal becomes blocked on its own only after three consecutive execution failures (accounting.rs:151). Continuations with zero tool calls do not count toward anything.
  • continuation.md:24 defines a verified wait as polling "a specific process, session, job, or tool handle confirmed live now". The cheapest behaviour that satisfies the prompt is one poll per continuation.
  • spec_plan.rs:1220 registers SleepHandler only when the model catalog lists clock (default SleepToolMode::ModelDriven). In models.json only gpt-6-astra has it. gpt-5.6-sol, gpt-5.5, gpt-5.4 and every custom-provider model get goal mode with no tool that waits without spending a turn.

Measurements

3,808 rollout files, July to September 2026, 59.2B input tokens, 97.8% cache hits.

  • 52 sessions (1.4%) had a goal active and used 29.2B of the 59.2B input tokens. The seven most expensive sessions in the archive all had a goal.
  • One hour of waiting at 160–245K context: 83–188M input tokens in a goal loop without sleep; 10–15M on gpt-6-astra with clock.sleep (60 and 129 calls of 45 s); about 0.5M if the wait were event-driven.
  • Empty continuations: chains of four goal turns with zero tool calls inside 20 seconds, each rereading 65K+ tokens. In one session 14 such turns cost 77M input tokens.
  • Worst session: 173 continuations in 48 minutes, 466 model calls, 149M input tokens, stopped by a 429 from the provider.
  • The weekly limit on a Pro account reached 99% fifteen times in ten weeks, median 23 h after the window opened.

The numbers can be reproduced on any account with codex-rollout-audit, one Python file that reads ~/.codex/sessions locally. Full write-up with charts: https://relux.works/en/blog/codex-goal-token-burn/

Requested changes

All four keep the goal moving; they remove the idle work between moves.

  1. Back off before the next continuation when the previous one made no tool call or only polled, for example 15 s doubling to a cap, instead of restarting at once.
  2. After N consecutive continuations with zero tool calls, move the goal to blocked with a message, the way three execution failures do today.
  3. Register clock.sleep whenever a goal is active regardless of the catalog, or make sleep_tool.mode = "always_on" the default when goals are enabled.
  4. In continuation.md, name clock.sleep and a long empty write_stdin as acceptable forms of a verified wait.

What steps can reproduce the bug?

  1. Codex CLI 0.154, ChatGPT Pro, any model without clock in the catalog, e.g. codex -m gpt-5.6-sol. Keep sleep_tool at its default (ModelDriven), so clock.sleep is not registered.
  2. Start something that finishes in about ten minutes and cannot be observed by event, e.g. in the session: (sleep 600; touch /tmp/done) &.
  3. /goal wait until /tmp/done exists, then print its timestamp and stop.
  4. Leave the session alone for ten minutes.
  5. Observe: a new continuation turn starts every 10–20 s, each one runs a single exec_command (ls /tmp/done, sleep 30, tail on a log) and ends; the next continuation starts within 0.03 s. Some turns contain no tool call at all and are still followed by an immediate restart.
  6. After the goal completes, inspect the rollout: python3 codex_rollout_audit.py session ~/.codex/sessions/<date>/<rollout>.jsonl --commands. The gap_s column shows the 0.02–0.05 s restarts, in_M shows the full context resent on every continuation, and the summary line reports total input tokens spent while waiting for one file to appear.
  7. Repeat with gpt-6-astra: the model calls clock.sleep, continuations are an order of magnitude fewer and the same ten minutes cost 10–15M tokens per hour instead of 83–188M.

What is the expected behavior?

A goal that is waiting for an external process should not cost more than a few model calls per hour. After a continuation with no tool call, or one that only polled, the next continuation should start after a growing delay rather than 0.03 s later, and repeated empty continuations should end in blocked with a message. Any model with an active goal should have clock.sleep available, and the continuation prompt should accept it as a verified wait.

No response

Additional information

No response

Activity

  1. added
    CLIIssues related to the Codex CLI
    agentIssues related to the core agent loop
    rate-limitsIssues related to rate limits, quotas, and token usage reporting
    on Sep 11, 2026
  2. github-actions commented on Sep 11, 2026

    @github-actions
    Contributor
  3. ForkAroundAndFindOut commented on Sep 11, 2026

    @ForkAroundAndFindOut

    Looked more closely at #23094. It appears related but distinct: that PR stops goal continuation for true blocked/usageLimited states, whereas this issue concerns legitimate waiting while the goal remains active, where repeated continuations/polling can still burn context and tokens.

    So #23094 provides some of the state machinery that could help here, but doesn't appear to address the active-wait loop itself.

  4. denbrazhko commented on Sep 11, 2026

    @denbrazhko

    Adding another real-world data point that seems consistent with the excessive context/continuation cost described here.

    On ChatGPT Plus using GPT-5.6 Sol Medium, a recent Codex task consumed roughly 20% of my 5-hour quota. The task was mostly repository analysis: inspect GitHub Actions in two Flutter repos, inspect how an existing Telegram bot receives build status, and determine the infrastructure needed to add TestFlight/App Store deployment status via Apple webhooks. I did not ask Codex to implement the solution.

    The striking part is the useful-work comparison. Roughly 20% of a Claude 5-hour window was enough for me to analyze a Figma file with 150+ frames, produce a restructuring plan, then continue for about seven more turns designing and implementing an automated Flutter UI/navigation testing tool, including code and GitHub Actions work.

    Obviously these quotas cannot be compared 1:1. But this is not a small efficiency difference; it is close to an order-of-magnitude difference in useful work in my workflow, and current Sol Medium consumption is also much worse than my previous Codex experience.

    If repeated continuations are causing Codex to repeatedly process/account for large portions of the accumulated context, that would fit what I’m observing: relatively modest analysis tasks become disproportionately expensive as the session progresses.

    It would be very useful if the client exposed a per-turn breakdown of input, cached input, output/reasoning, tool/context overhead, and the corresponding quota charge. Right now the 5-hour percentage can drop dramatically without giving the user enough information to identify what actually consumed it.

  5. etraut-openai commented on Sep 12, 2026

    @etraut-openai
    Contributor

    Thanks for the detailed analysis. We recently added a circuit breaker that blocks a goal after three consecutive automatic turns with empty final answers and no other activity. This change is newer than 0.154.0 and should address the empty-final loops you identified.

    It does not address the broader waiting cost: tool calls or nonempty responses reset the counter, so repeated polling can still keep a goal active. The goal continuation loop is intended to make the agent extremely "gritty", but it does allow the agent to pause the goal if it determines that it's truly at an impasse. You can also adjust the objective to include other reasons for the agent to stop (e.g. "stop if you have polled for more than five minutes").

    If the problem occurs again, please run /feedback from the affected session, include the session diagnostics, and share the feedback ID here so we can investigate the remaining behavior.

  6. FINDarkside commented on Sep 14, 2026

    @FINDarkside

    Why does the goal need to be paused? I think this is a bug how Codex doesn't let the main agent wait for stuff when goal is active. While pausing the goal helps a bit (more on that later), it makes goals useless for any long running tasks that use subagents or external processes. I think codex just needs to stop waking up the main agent for no reason when it's either:

    • Waiting for some process
    • Waiting for subagent
    • Not explicitly waiting for either, but it'll still get notified when either completes

    I burned my $200 plan weekly limits in 8 hours when Astra spent 100m tokens per hour doing nothing when subagents with 5.6 Sol were working.

    Even when the goal is paused, Codex will keep burning insane amount of tokens by then polling the subagent status with wait_agent which for some reason defaults to 30s timeout. You can of course manually instruct it to use 25 minute timeout or whatever, but I think this should be part of harness. When pressed about why it needs to poll, instead of just waiting for signal when the agent completes, 5.6 Sol high responded with

    Agent messages are delivered automatically, but keeping my turn open until one arrives requires that wait call. If I end with a final answer, I’m no longer waiting. With the goal paused, I shouldn’t assume I’ll automatically resume when an agent finishes.

  7. icekinder commented on Sep 24, 2026

    @icekinder

    Adding a desktop trace of the active-wait case. On 2026-09-22, one Goal task was waiting for an existing scheduled wake. The native runtime delivered Goal continuation prompts at 01:36:19, 01:36:37, 01:36:50, and 01:37:02 UTC. Each turn made zero tool calls, returned a nonempty answer saying to wait, and was followed by another automatic continuation. A fifth prompt arrived at 01:37:11 UTC. There was no intervening direct user message in the recorded task turns.

    This is narrower than an empty-final circuit breaker. The final answers were nonempty, but there was no new work or external state change. The official Goals guide says a no-tool continuation suppresses the next automatic continuation. The observed sequence appears inconsistent with that description. The local task record does not expose the native dispatch reason, so I cannot rule out an unrecorded internal trigger.

    In a separate 24-hour local audit, 34 Goal continuation episodes used only waiting or coordination tools and eight made no tool call. Those 42 episodes contained 112 model responses. Most Goal continuations did use other tools, so I am not labeling all continuation activity as waste. The repeated no-tool sequence was not observed in a later, quieter 24-hour window.

    Please consider a durable active but deferred Goal state that resumes once on the scheduled event or meaningful new input. Automatically marking the Goal blocked is a poor substitute where blocked is terminal and may release task ownership. I have kept private task identifiers, paths, and full logs out of this comment. I can provide a sanitized trace through a private support route if useful.

  8. Keesan12 commented on Sep 26, 2026

    @Keesan12

    173 continuations in 48 minutes on a goal that was clearly looping is brutal. The agent convinces itself it hasn't converged when it has, and just keeps going.

    We track whether each continuation is delivering new output or retreading the same ground. When it's the latter, the run stops. Built into martin-loop — open source.

  9. ivanopcode commented on Sep 26, 2026

    @ivanopcode
    Author

    @etraut-openai

    Thanks for the detailed analysis. We recently added a circuit breaker that blocks a goal after three consecutive automatic turns with empty final answers and no other activity. This change is newer than 0.154.0 and should address the empty-final loops you identified.

    It does not address the broader waiting cost: tool calls or nonempty responses reset the counter, so repeated polling can still keep a goal active. The goal continuation loop is intended to make the agent extremely "gritty", but it does allow the agent to pause the goal if it determines that it's truly at an impasse. You can also adjust the objective to include other reasons for the agent to stop (e.g. "stop if you have polled for more than five minutes").

    If the problem occurs again, please run /feedback from the affected session, include the session diagnostics, and share the feedback ID here so we can investigate the remaining behavior.

    Thanks for clarifying the scope of the circuit breaker. The remaining active-wait behavior needs a runtime-level solution.

    First, there is a discrepancy with the official Goals documentation, which states:

    If a continuation turn makes no tool call, the next automatic continuation is suppressed so Codex does not spin.

    The described circuit breaker is narrower: nonempty responses reset its counter even when no tools were called and no progress occurred. The September 24 report in this thread describes precisely that sequence. Could you clarify which behavior is intended?

    Second, efficient waiting is a basic requirement for long-running goals. If a CI run takes an hour, invoking the model with a large context every 30 seconds merely to discover that CI is still running is unreasonable. Repeatedly confirming unchanged external state does not advance the goal.

    Users reasonably trust the harness to manage this. Cached input tokens are discounted, not free, and users in this thread report substantial subscription quota consumption while the parent agent is waiting. They should not need to understand the scheduler or write defensive prompts to prevent that.

    Adding “stop after five minutes of polling” limits the damage, but leaves the original requirement unmet: wait for the process and then continue autonomously. A legitimate dependency taking longer than five minutes should not require abandoning the goal or asking the user to restart it.

    My request is straightforward: the goal loop should support suspending model execution while preserving the goal and automatically resuming when relevant work completes or new input arrives. This must work for processes whose completion time is unknown.

    Where completion notifications are unavailable, the harness should provide bounded polling with sensible backoff, without requiring a model inference for every unchanged status. A watchdog can still detect stalled work.

    Could you confirm whether this active-but-waiting behavior is considered a supported requirement, and whether the remaining issue will be tracked beyond the empty-response circuit breaker?

  10. xxxupeng commented on Oct 5, 2026

    @xxxupeng

    I ran into the same issue with long-running training jobs, so I made a small workaround:

    https://github.com/xxxupeng/skills

    The scheduled-autonomy skill is basically Goal-like autonomy with sparse host-side wakeups: Codex keeps working toward the same objective, but when there’s nothing useful to do, it ends the current turn and uses a systemd timer to wake the same thread later instead of polling continuously and burning tokens.

    Feel free to try it, and issues or feedback are very welcome.

  11. KZ8589 commented on Oct 9, 2026

    @KZ8589

    Goal keeps auto-continuing after one failed tool and tool-free blocker reports

    This adds a bounded reproduction of a failure followed by nonempty, tool-free reports on a newer version to the related continuation behavior reported in this issue. It does not independently verify the original author's measurements.

    Observed behavior

    On the official Codex CLI reporting version 0.162.0-alpha.2, an active synthetic goal continued after one failed file read followed by nonempty, tool-free reports of the same blocker. Four turns completed and a fifth started before the bounded test driver ended the experiment. This is a bounded counterexample, not a claim of an observed infinite execution.

    The model did not repeat the file read or invent a successful result. The problem was the host continuing to schedule more reports without a changed condition.

    Reproduction conditions

    1. Use a disposable directory with an intentionally absent text file and a fresh synthetic goal whose completion requires reading that file. Do not use an existing work goal.
    2. In the initial turn, ask for one read attempt, an accurate report of failure, and no repeated read or alternate search. For this host-isolation experiment, subsequent turns must report from the existing evidence without calling additional tools, including lifecycle tools.
    3. Observe turn boundaries, actual command exit codes, goal state, and the absence of subsequent tool executions.
    4. Bound the experiment to a small number of turns, then stop the test host. Do not leave an unattended loop running.

    This intentionally restrictive prompt isolates host behavior. In the separate ordinary-contract control, lifecycle tools remained allowed and the model used the supported blocked transition after repeated blocking conditions. That control passed. The restrictive test should not be described as ordinary recommended usage.

    Source findings

    The matching public tag points to commit 74e804d. Version-string agreement does not establish reproducible-build identity with the distributed binary.

    In codex-rs/ext/goal/src/accounting.rs, nonempty assistant output counts as activity and resets the empty-turn counter. The execution-failure counter separately requires a failed execution in that turn. One failed read followed by nonempty reports with no executions therefore falls between the two conditions. The goal stop extension consults those counters before the next dispatch.

    A read-only comparison at public main commit 0ada5d8 (2026-10-09) found accounting.rs, runtime.rs and extension.rs byte-identical to the prior 2026-10-08 main observation for this investigation.

    The Goals guide describes suppression after a continuation without a tool call. The observed installed behavior and the counters above appear narrower than that description.

    Local repair experiment

    An isolated candidate remembers an actual failed execution and blocks the unchanged goal after three consecutive automatic turns without tool activity, emitting a warning and preserving the objective. A successful tool, new user input or goal change clears that pending failure. Text-only work without a preceding failure remains allowed.

    Evidence: 43 accounting/backend tests, 8 actual app-server tests using scripted local SSE, and packaging checks passed. In a separate live-model run, one failed read followed by three tool-free automatic turns resulted in blocked, one warning event, and 30.095 seconds without another turn. A two-phase normal task continued and completed. These are bounded tests, not a guarantee for every loop or model decision.

    The heuristic can stop useful prose after an unresolved execution failure. It does not catch every kind of no-progress loop, including repeated successful but unhelpful tool calls. Its tradeoff needs maintainer review.

    The local candidate is not recommended as a Desktop replacement: the installed Desktop rejected its app-tools peer at code-signing authorization. The official executable was restored and app-tools functionality recovered. No signature checks or security settings were weakened.

    Requested outcome

    Please clarify the intended tool-free continuation contract and either close this host-side gap in an officially distributed build or document a supported way to enforce a bounded failure/report sequence without replacing the signed Desktop runtime. A reviewed source patch and deterministic tests are available locally; they have not been published with this draft.

    No credentials, personal paths, existing conversation identifiers, business data, or complete environment configuration are included here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIagentIssues related to the core agent loopbugSomething isn't workingrate-limitsIssues related to rate limits, quotas, and token usage reporting

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions