Repository navigation
Feature request: cost-aware adaptive orchestration (single → cascade → critique) for Codex #45362
Description
What variant of Codex are you using?
Codex App / CLI / IDE / Web. This is primarily a runtime/orchestration proposal that could be shared across Codex surfaces.
What feature would you like to see?
I would like Codex to add an opt-in cost-aware adaptive orchestration mode whose goal is:
Use the least expensive amount of inference that can reliably complete the task, and spend stronger-model compute only when evidence justifies it.
This is broader than automatic model selection. Instead of choosing one model for an entire task, Codex could dynamically choose an execution pattern per task or phase.
A useful first version could support three bounded strategies:
- Single — route straightforward work directly to the cheapest model/reasoning level expected to meet the quality bar.
- Cascade — let a cheaper/faster model attempt the work first, validate the result, and escalate to a stronger model only when the candidate fails a quality gate.
- Critique — for work where independent review is valuable, let one model produce the candidate, use an isolated read-only critic to identify concrete problems, then allow one bounded revision pass.
Conceptually:
user task
|
v
cost/quality router
|
+-- Single ------> efficient model --------------------> result
|
+-- Cascade -----> efficient model -> quality gate
| | pass -> result
| | fail
| v
| stronger model -------> result
|
+-- Critique ----> solver -> isolated critic -> revision -> result
Codex already has many of the useful primitives: multiple model/reasoning tiers, subagents, model overrides, tests/tools, persistent task state, and increasingly capable orchestration. The missing piece is a product-level policy that optimizes the whole task for outcome quality, token/plan usage, latency, and compute rather than requiring the user to manually decide every model transition.
Important design constraints
1. Optimize cost per successful task, not token count in isolation.
A weaker model that repeatedly fails, re-reads the repository, or spawns many agents can cost more than one strong pass. The router should consider expected retries, context duplication, verification cost, and probability of success.
2. Routing itself must be cheap.
Do not require a full expensive model turn merely to decide which model should handle the task. Prefer runtime/task metadata, lightweight classifiers, cached historical signals, or a very small routing model where appropriate.
3. Escalate from evidence.
Useful escalation signals could include:
- failing tests or validation;
- repeated unsuccessful tool cycles;
- low-confidence/ambiguous changes;
- unresolved compiler/type errors;
- security or migration risk;
- critic findings;
- evidence that the current task boundary is wrong;
- explicit user quality/budget requirements.
4. Avoid multi-agent overhead when it is not worth it.
Every additional agent may duplicate system instructions, repository context, tool schemas, and discovery work. Fan-out should happen only when predicted benefit exceeds this fixed context cost. Context/cache reuse should be treated as part of the optimization objective.
5. Independent review should be isolated and bounded.
A critic should normally be read-only/tool-less and should not be able to modify the repository. One critique + one revision is a reasonable initial bound. On cancellation or failed validation, Codex should avoid leaving a partially applied patch.
6. Account for the complete workflow.
Expose total usage across routing, drafting, critique, revision, escalation, retries, and subagents. A strategy should not be considered cheaper merely because one individual call was cheaper.
7. Preserve user control.
A pinned model/reasoning level should remain authoritative unless the user explicitly enables adaptive orchestration. An experimental mode could offer policies such as:
Adaptive: Balanced
Adaptive: Minimize usage/cost
Adaptive: Maximize quality within budget
Manual
The UI should make the effective strategy observable without exposing chain-of-thought, for example:
Strategy: Cascade
Initial: lightweight model / medium
Validation: tests + runtime quality gate
Escalated: no
Estimated usage saved vs selected strong baseline: 58%
Reference: GitHub Project HydraFusion
GitHub recently published a useful reference implementation/research direction:
HydraFusion selects among Single, Cascade, and Critique workflows and treats workflow selection as an optimization problem across quality, cost, and latency.
GitHub reports the following controlled offline results relative to Claude Opus 5:
| Benchmark | Estimated cost | Verified quality |
|---|---|---|
| TerminalBench 2.1 | 67% lower | +4.9 points |
| DeepSWE | 36% lower | -1.5 points |
| CheckpointBench | 65% lower | -0.1 points |
Those numbers are specific to GitHub's evaluated model pool, benchmark versions, and pricing assumptions, so I am not claiming Codex would reproduce them. They do demonstrate that runtime orchestration is worth evaluating as a way to approach frontier quality at materially lower average workflow cost.
The point is not to copy GitHub's implementation. Codex has its own architecture and could potentially do better by combining adaptive model routing with its existing subagent/task infrastructure.
Why this matters to users
For many users, the practical limit on long Codex sessions is not model capability but usage/cost. If routine repository search, exploration, test triage, straightforward edits, and review can use cheaper inference while difficult reasoning escalates only when needed, users should be able to complete more useful work within the same budget or plan allowance.
This could make long-running coding workflows substantially more accessible while preserving frontier-model capability for the portions that actually need it.
Compute and environmental efficiency
There is also a broader infrastructure benefit. Systematically avoiding unnecessarily expensive inference should reduce avoidable accelerator work, electricity consumption, cooling demand, and datacenter load per completed task.
The carbon impact should not be overstated: emissions depend on hardware utilization, datacenter energy mix, caching, workload displacement/rebound effects, and other factors. But cost per successful task and compute per successful task are measurable optimization targets, and reducing unnecessary inference is directionally aligned with lower energy demand.
At Codex scale, even modest improvements in average compute per successful task could be meaningful.
Suggested evaluation / acceptance criteria
I would evaluate this mode against a fixed strong-model baseline using real agentic coding workloads.
Track at least:
- verified task completion rate;
- total input/cached/output tokens;
- normalized plan usage or estimated API cost;
- wall-clock latency;
- number of model legs and retries;
- duplicated context sent to subagents;
- escalation rate;
- tests/validation success;
- failures caused by under-routing;
- cost per successfully completed task.
A successful policy should reduce average usage/cost materially while preserving an explicit quality floor, rather than merely routing everything to smaller models.
A safe rollout could begin as an experimental opt-in with shadow evaluation before automatic routing is enabled broadly.
Related Codex requests
This seems complementary to existing requests rather than identical to them:
- Add an explicit Smart Mode for automatic model and reasoning-effort selection #34388 — Smart Mode for automatic model and reasoning-effort selection. This focuses primarily on selecting a model/effort; this request adds compound per-task execution strategies, quality gates, critique, and conditional escalation.
- Proposal: Codex-native evidence-driven semantic escalation for dynamic multi-agent graphs #40037 — evidence-driven semantic escalation in dynamic multi-agent graphs. This provides useful primitives/signals for deciding when execution should be retried, escalated, or replanned.
- Subagent fan-out can increase usage because each agent pays fixed context/tool/skill overhead #39808 — notes that subagent fan-out can increase usage because every worker pays fixed context/tool/skill overhead. A cost-aware orchestrator should explicitly account for this.
- Feature request: Pre-turn routing hook for plugins to select model and reasoning effort #40599 — pre-turn routing hook for plugins. A native implementation could use similar task signals without requiring an extra routing turn.
- Support cost and budget metrics in TUI status_line #38721 — cost/budget metrics. Better accounting would make adaptive policies auditable to users.
I would be happy to test an experimental implementation and compare it against fixed-model Codex workflows on real repository tasks.
Activity
github-actions commentedon Sep 14, 2026
Potential duplicates detected. Please review them and close your issue if it is a duplicate.
Powered by Codex Action
pkwow896-beep commentedon Sep 23, 2026
Engineering follow-up: accuracy-first automatic routing for subscription users
Adding an implementation-oriented user proposal and a small measured observation to this existing request instead of opening a duplicate. Submitted with the user's authorization; this feedback was organized with Codex assistance. Related user-facing need: #46366.
Problem and requested product outcome
A nondeveloper using Codex for long-running data reconciliation and software work cannot reliably decide which model and reasoning level will avoid omissions or incorrect judgments. This user keeps Astra at Ultra as a precaution, reports that sustained work can exhaust their subscription allowance within a day, and wants more verified work from that allowance without taking responsibility for model expertise.
The request is native, opt-in, accuracy-first automatic model AND reasoning selection within the same task, not a cheaper default, a mandatory low-model first attempt, a plugin that adds routing turns, or a requirement for the user to pick models manually.
Optimize total subscription usage per correctly completed task, including controller, handoffs, verification, escalations and rework. Preserve an explicitly evaluated quality floor. A user should be able to specify the outcome, allowed actions, desired quality and spending/time preference; Codex should own model selection.
Proposed minimum product contract
Simple controls. Offer "Auto: accuracy first" alongside manual/pinned selection. Optional daily/task usage budget and urgency should not require knowing model names. Manual pins and tool/write approval policies remain authoritative.
Route bounded work units, not just whole conversations. Assess modality, ambiguity, cross-source identity/linkage, dependency depth, irreversible impact and available verification evidence before choosing a supported model/effort. Large routine accounting may be easier than one ambiguous identity correction. Risky work can go directly to a stronger model; do not force a cheap failed attempt first.
Escalate on evidence. Examples: failed invariants, source contradictions, unsupported joins, inadequate coverage, security-sensitive change or a concrete review finding. Do not use model self-confidence alone. A missing fact, invalid permission or unavailable service is not solved merely by adding another model.
Preserve task continuity. Maintain one stable goal/work-unit identity and a versioned handoff containing authoritative instructions, permissions, source/evidence references, completed outcomes, unresolved items and the exact next action. A handoff must not forget constraints, report partial completion as global completion, repeat successful writes, or reopen unchanged completed work. Invalidate only evidence actually affected by a change.
Use independent checks where feasible. Prefer deterministic reconciliation, focused tests and direct runtime/readback evidence to blanket second-model review. For genuinely unresolved semantic risk, permit bounded stronger-model review of the disputed unit. No unlimited cascade, duplicate full-task execution or automatic debate among all models. If reliable evaluation is unavailable, retain the stronger model or label the uncertainty rather than claiming preserved quality.
Subscription-aware, not just token-aware. Use the account's available models, included usage, reset windows and applicable credit weights. Unknown usage must be shown as unknown. When the quality requirement cannot fit the remaining budget, offer an explicit choice such as wait, narrow scope or use separately authorized credits. Never silently lower the quality policy, purchase credits, enable overage or substitute a different provider.
Make routing auditable without exposing chain-of-thought. Show requested/effective model and effort, a short reason, verification performed, unresolved risk, and measured usage. Include all agent/controller legs, cached and uncached input, output and reasoning breakdown without double-counting reasoning already included in output. Separate measured usage, estimates and account-billed credits. Do not present an unrun expensive baseline as measured savings.
Suggested runtime implementation boundary
Use a shared Codex runtime policy across App/CLI/IDE rather than a prompt-only workaround. At an eligible work-unit boundary:
task state -> capability/risk assessment -> budget/quality constraints -> model+effort -> execution -> evidence validation -> accept, targeted escalation, or explicit holdPersist a small routing event with work-unit ID, policy version, effective configuration, reason category, evidence status and usage totals. Reuse valid context/cache and evidence; switching models should not itself create a user-visible task or replay all history into several agents.
Apply an overhead budget to routing. Use deterministic/cached signals when sufficient, and bypass adaptation when estimated savings do not exceed routing and handoff cost. Treat cancellation, context compaction, unavailable models and quota exhaustion as explicit state transitions. Tool permissions must not expand on escalation, and retries must not duplicate side effects.
Small local observation, with important limits
Observed on Windows, Codex app 26.917.51856 (build 10492), bundled CLI 0.155.0-alpha.16, on 2026-09-23. Two fresh ephemeral runs used the same PII-free 64-record accounting packet, response schema and high reasoning setting; one generation request per model, zero tool calls and no retries. A predeclared 37-assertion oracle covered accounting and refusal to infer unsupported global totals/completion.
Cached input was zero. Estimates use the published Codex standard credit rates: input/output credits per million of 50/250 for Sol and 250/1250 for Astra, as checked that day.
This was about 77.76% lower estimated generation credits, but Sol used 288 more total tokens and was only 3 seconds faster. That distinction matters: token reduction, subscription savings and throughput are not interchangeable.
This is one narrow paired sample, not statistical evidence of general equivalence or an Astra Ultra comparison. It excludes image interpretation, code changes, live DB integration, the original long conversation, controller/preparation/scoring overhead and account billing readback. The CLI records the explicit requested models but did not independently attest a server-canonical runtime model. No raw records or conversation are attached. The result justifies evaluating bounded routing; it does not establish that Sol can replace Astra for this user's complete workflow.
Acceptance criteria and release evaluation
Evaluate on representative end-to-end tasks, not only answer quality: code changes with tests/runtime proof, visual extraction, cross-source reconciliation, long-context instruction retention, and partial-versus-global completion. Include multilingual/nondeveloper users.
Before rollout, define a preregistered non-inferiority margin for verified completion and omission/incorrect-link rates against an appropriate pinned strong-model baseline. Treat unauthorized writes, lost mandatory constraints and false-completion regression cases as release blockers. Universal zero-error guarantees are not realistic; make observed failure rates and uncertainty visible.
Report total credits per verified completion, task completion/coverage, critical error and omission rates, correction/rework rate, controller overhead, elapsed time and escalation frequency. Savings are useful only if end-to-end quality remains within the agreed bound.
Start with internal/offline evaluation, then a small opt-in cohort with routing explanations and an immediate manual fallback. Passive shadow decisions should not trigger duplicate paid generations by default. Any paired experimental generations should be explicitly budgeted and consented to.
Requested follow-up
Please triage this as a native Codex runtime + subscription-UX feature and link it to the relevant tracking work if it already exists. A useful response would clarify whether an accuracy-first Auto mode is planned, which quality/usage metrics would gate it, and whether there is a supported pilot path. The user is asking OpenAI to own this product decision, not for another manual model-selection guide.
pkwow896-beep commentedon Sep 23, 2026
Additional case evidence and release gates: do not infer whole-task reliability from a few passing model answers
Adding a distinct observation from another long-running Codex software/operations task to the detailed accuracy-first proposal above. This is submitted with the user's explicit authorization and organized with Codex assistance. It supplements the existing proposal rather than opening a duplicate.
User need: A nondeveloper uses Astra Ultra because they cannot assess omissions or wrong decisions themselves. They want Codex to choose the model, reasoning effort and any delegation needed for the work, using their subscription efficiently while preserving an evaluated quality requirement. Asking them to decide when a smaller model is adequate transfers the difficult judgment back to the person least able to assess it.
1. What this task exposed
The assistant initially recommended keeping Astra as owner after a small comparison, then recommended Sol 6 Ultra as the whole-task default after another small comparison. The evidence did not establish either model's long-horizon, autonomous reliability sufficiently to justify a categorical replacement recommendation.
The useful distinction was correct implementation versus correct choice of the next action:
Limits are essential: this was a tiny purposive sample, not a population error rate or a production reliability comparison. Task-prompt bytes matched within each case, but runtime context differed. It used CLI 0.156.0 on Windows and supplied-context JSON answers; tools and subagent calls were explicitly prohibited. Consequently, the configured Ultra runs did not exercise the full desktop Ultra delegation workflow. They did not measure autonomous browsing/deployment, actual charged subscription credits, or Astra Ultra end-to-end performance. Qualitative grading included owner judgment and a blind review of strategy excerpts, not a large independent benchmark. No private customer data, project identifiers or raw transcript are attached.
This supports evaluating native routing; it does not demonstrate that Sol can replace Astra throughout this user's work.
2. Incremental product requirements
A. Optimize under a quality constraint, not by model rank or remaining quota alone.
Offer one understandable “Auto — quality first” entry point. Route only among models/efforts actually entitled and supported for this account/client/tools/context. Treat existing explicit pins as authoritative until the user enables Auto. Do not force a cheap first attempt for work with consequential uncertainty. Roll out by task category; retain the qualified stronger route when the evidence for down-routing is insufficient.
B. Reassess at real work boundaries.
Understanding the objective, implementing a confirmed repair, and deciding whether evidence supports completion are different units. A passing code test must not authorize cheaper strategic decisions automatically. Permit escalation for contradictory requirements, unsupported consequential assumptions, repeated failed checks or insufficient coverage. Missing external facts/authentication need the fact or authorization, not an endless model ladder.
C. Preserve state and verification capacity across every switch.
Keep objective, user corrections, allowed actions, evidence provenance, unresolved items and successful external-action identities. Do not restart a task, repeat an external write or weaken completion criteria during a switch/compaction/resume. Reserve budget for required tests and readback before spending the whole allowance on implementation. If the quality-qualified route or verification budget becomes unavailable, preserve progress and explain the real choice; never silently downgrade or assert completion.
D. Use real account accounting.
Different plans, included-usage windows, credit balances, workspace restrictions and reset times require server-authoritative entitlement/accounting. Published token-credit prices alone do not determine included subscription consumption. Measure the full workflow: routing overhead, cached/uncached context, all agents, retries, escalation, verification and rework. Show unknown usage as unknown, and distinguish estimates from metered debits. Do not buy credits, enable overage or redeem a reset automatically.
E. Keep the interface simple and auditable.
Show short operational reasons, for example “Using a stronger model to resolve conflicting requirements,” with requested/effective model and effort available in details. Expose actual runtime metadata rather than the model's self-description. Users can retain a manual pin or set a spending/time preference without needing model expertise. Routing must not alter permissions.
3. Suggested minimal implementation boundary
This is a proposed host/runtime policy, not an assertion about undocumented internals.
The App Server documentation already describes model/effort overrides in
turn/start; it also says overrides persist as later defaults andturn/steerdoes not accept them. A native implementation therefore needs an explicit distinction between the user's pinned/default setting, Auto policy, and the effective work-unit route. A temporary automatic escalation must not silently rewrite a manual preference.An initial release could select a qualified route at turn start and use an explicit safe continuation checkpoint for later changes. Do not synthesize a second user task merely to ask which model should handle the first, or interrupt/replay an in-flight tool action to change routes. Reuse existing runtime state and verification mechanisms rather than making users install a routing plugin or extra hooks.
Suggested internal event fields, not a proposed existing API: work-unit ID, policy version, entitlement snapshot ID, requested/effective model+effort, reason category, quality-gate evidence references, budget reservation and reconciled usage. Store concise operational metadata, not chain-of-thought.
4. Acceptance tests that would make this actionable
Evaluate full, long-running, multilingual tasks with changes of direction and missing evidence, not just isolated answers. Compare against a predeclared strong configuration actually available to each evaluated plan. Predeclare quality non-inferiority margins and sample sizes by task category; include omissions, consequential decision errors, unsupported completion, corrective user turns and rework. Treat unauthorized writes and lost mandatory constraints as release-blocking regression cases. Report uncertainty rather than a zero-error guarantee.
Use an internal/offline phase, then an opt-in pilot. Passive shadow recommendations should not generate a second billed execution by default. Measure total credits/allowance per verified task, quality and latency together; roll back any routing category that violates its quality criterion.
Requested triage: Please route this to Codex runtime/model-routing and subscription-UX ownership, and link the relevant tracking work if it exists. Is an accuracy-first Auto mode being considered, what end-to-end quality/usage gates would be required, and is there a supported pilot route? The requested outcome is for Codex to own this choice, rather than another guide requiring nondevelopers to become model-selection experts.
References checked on 2026-09-23: official feature-request route, current model/Ultra controls, subscription/credit accounting, and App Server.