Skip to content

Suspected degradation of gpt-6-astra: premature turn termination (~30s), completion reports for work that was never done, "I guess..." instead of investigating #43329

Open

Description

@hwb96

Summary

On 2026-09-06 (13:00–16:30 UTC+8) I hit a sustained, consistent pattern of degraded agentic behavior from gpt-6-astra. The failures are not "wrong answers" — they are failures of agent behavior itself: the model narrates work instead of doing it, terminates turns prematurely, and reports completion falsely. The session became unusable and I switched that workload to a competitor model mid-afternoon.

Environment

  • Clients tried, in order:
    1. Orca 1.4.195 (Codex desktop GUI driving codex app-server)
    2. codex-cli 0.153.4 in ghostty terminal — suspected Orca first, but the same behavior reproduced here
    3. Codex inside the official ChatGPT app (GUI) — still reproduces
  • Model: gpt-6-astra, model_reasoning_effort = "high", service_tier = "priority"
  • Provider: stock OpenAI backend (requires_openai_auth = true, wire API responses, no custom base URL — no third-party relay involved)
  • Timezone: UTC+8; bad window 2026-09-06 ~13:00–16:30 local
  • Task: complex debugging in a large Python codebase (document parsing / OOXML)

Symptom 1: turn ends in ~30s with a completion report that cannot be real

Timestamped log (task: implement a new flowchart-parsing stage, then run regression):

  • 11:52:42 user: start implementing X, then run regression, verify output
  • 11:52:47 model (5s later): plan statement
  • 11:53:09 model (22s later): "code changes have landed on branch …, compile check and diff check passed … not pushing/deploying yet", then the turn ends.

The claimed work (locating drawing-group handling across the codebase, wiring it into the figure pipeline, running compile + diff checks) requires dozens of tool calls and minutes at minimum. 22 seconds cannot contain it — even if tool calls were hidden in the UI, the wall-clock time is physically insufficient. Moreover, the "completed" list in that message consisted of items from the previous task; the new task was described only in future tense ("I will reuse …"). Zero work was started, wrapped in a full status-report format.

Tell: it also wrote "I haven't fabricated results for the flowchart part yet" — a model that is actually working does not talk about fabrication.

Symptom 2: partial completion reported as full completion

I list 4 explicit, independent bugs. It fixes (part of) one and reports the whole batch done. Earlier variant: fixes a sliver, stops, and asks "if you need, I can continue" — pushing orchestration back to me. Both forms make its completion reports unusable: I now have to re-verify everything it claims.

Symptom 3: "I guess…" instead of investigating

On a bug it does not read the repo code, does not check logs, does not run anything — it starts from "I guess it might be …" and patches the guess. Result: poor generalization; the same indentation class of bug has "been fixed" repeatedly and keeps coming back, consistent with surface patches on unverified root causes.

Symptom 4: the capability is demonstrably still there

When I explicitly instruct "do NOT change code, do a root-cause analysis first", the same model immediately explores properly for 5–10 minutes: reads real code, forms and discards hypotheses. So this is not a missing capability — the default stop/evidence calibration is what breaks.

The client is ruled out

I suspected the Orca GUI at first, so I switched twice: first to the plain codex-cli 0.153.4 in a ghostty terminal, then to the official Codex UI inside the ChatGPT app. The behavior is identical in all three — same premature termination, same fabricated completion reports, same "I guess…" starts. Whatever is wrong, it is upstream of any client.

Why I suspect load-related degradation

  • Same repo, same prompts, drastically different behavior across sessions (intermittent, not permanent).
  • The degradation persisted through an entire afternoon session, then I stopped testing — so I can't say whether it recovered.
  • I understand this is anecdotal; that's part of why I'm posting.

Questions

  1. Others: are you seeing 20–30s premature terminations with plausible-sounding but physically impossible completion reports on gpt-6 models?
  2. OpenAI: is there any routing/fallback path that can serve a degraded (smaller/quantized/heavily-cached) variant under high load, even on priority tier? That would explain both the intermittency and the specific failure shape — narration replacing tool calls is exactly what a weaker model produces.
  3. Any mitigation available (model pinning, region, off-peak)?

Activity

  1. added
    bugSomething isn't working
    model-behaviorIssues related to behaviors exhibited by the model
    on Sep 7, 2026
  2. Artisticleaf commented on Sep 7, 2026

    @Artisticleaf

    I observed the same phenomenon as you bro, and I‘ve been turning back to Sol with high effort, which might be over-engineering a little bit but actually gets the work done. I'd rather the agent spend a whole afternoon to solve the problem stupidly but not slacking off then announcing himself sota

  3. liubaicai commented on Sep 11, 2026

    @liubaicai

    same problem

  4. sun-jingtao commented on Sep 11, 2026

    @sun-jingtao

    Another data point: same-prompt, same-effort comparison before and after capacity throttling (Pro, macOS)

    I'm seeing the same pattern described here from gpt-6-astra: much less reasoning, no self-verification, and clearly weaker output. I have a like-for-like comparison from one account. This account was fully blocked by "Selected model is at capacity" on 2026-09-10 from 11:06 to about 18:00 (UTC+8) and has been intermittently blocked since. Every Astra run on 09-11 shows the drop.

    Task (Chinese prompt): "Create an HTML file with an SVG 2D animation of a pelican riding a bicycle." The 09-09 and 09-11 runs use the exact same prompt, which also asks to save the file to the Desktop and name it after the model. The 09-08 run has only the first sentence.

    Run (UTC+8) Effort Where Output tokens Reasoning tokens Wall time Result
    09-08 19:07 high Desktop, projectless 7,621 564 4m30s 14.4 KB. JS-driven pedaling, pause and speed buttons. Opened it in the in-app browser to check, then refactored.
    09-11 19:31 high Desktop, projectless 2,998 213 ~1m50s 5.8 KB. CSS-only animation, no controls. Wrote the file once and stopped.
    09-09 17:50 ultra Desktop, repo with AGENTS.md 14,124 1,620 7m52s 24.8 KB
    09-11 19:34 ultra Desktop, projectless 8,909 848 ~4m13s 7.6 KB
    09-11 19:52–20:00 ultra codex exec --ignore-user-config --ephemeral, 3 runs, 1 with the same AGENTS.md 5,568–6,503 431–1,116 2–4 min 8.8–11.4 KB

    Token counts are main-thread totals summed from the rollout token_count events. Both desktop ultra runs spawned one sub-agent, which is excluded on both sides. One of the three CLI runs was cut off by a capacity error after writing its file, so its tokens are not included.

    What I ruled out:

    • Reasoning effort. Every sampling request in logs_2.sqlite carries the selected effort: high for the high runs, ultra for the ultra runs. There is no silent reset to medium (compare Astra reasoning level frequently reset to Medium #44668).
    • Model field and fallback. Every sampling request is sent as gpt-6-astra. There are no fallback or model-switch events in the logs.
    • service_tier. It is not set in any of these runs, so none of them used priority.
    • Client. The core binary is codex-cli 0.153.4 in all runs. The desktop shell changed from 26.901.51231 to 26.903.71938, but the codex exec runs bypass the desktop app and the user config, and they show the same drop.
    • Context. The projectless desktop context on 09-08 and 09-11 differs only in plugin and skill list updates and a new "Inline Artifact Follow-Ups" formatting section. Adding the 09-09 repo's AGENTS.md back in a CLI run did not restore the output: 11.4 KB versus 24.8 KB.
    • Prompt. On the same day, gpt-5.6-sol at ultra with the exact same prompt produced a 21.9 KB file, in the same range as the pre-throttling Astra output.

    Caveats:

    Thread IDs:

    • 09-08 high, good: 01a080b3-baaf-74a1-912d-b9c29135ea27
    • 09-09 ultra, good: 01a08593-d43a-7b83-9844-d0dde12688e9
    • 09-11 high, degraded: 01a0903c-b518-7310-b242-ab1bac153be4
    • 09-11 ultra, degraded: 01a0903f-b7cf-7491-b331-15b8f3323a7c

    Could the team check whether these requests were served by the same model and the same reasoning budget before and after the 09-10 capacity event?

  5. tzf1003 commented on Sep 12, 2026

    @tzf1003

    same problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmodel-behaviorIssues related to behaviors exhibited by the model

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions