Skip to content

Session compaction fails with "context exceeds model limit" error #17340

Closed as not planned

Description

@he-who-is-not-him

Description

Description

Sessions can trigger the error:

"Session too large to compact - context exceeds model limit even after stripping media"

In my case, using a model with a 128k context limit, the session grew to 145,882 tokens. Notably, there hadn't been a user message in a while - only responses through the question tool accumulating in the conversation.

What I Think Is Happening

The overflow detection in isOverflow() (packages/opencode/src/session/compaction.ts:32-48) appears to use token counts from the previous turn's completed API response (lastFinished.tokens). This may not account for content that will be added in the next API call:

  1. New user message
  2. System prompts
  3. Reminders
  4. Tool schemas

This could create a timing gap where sessions exceed the safe threshold before compaction is triggered. However, I haven't fully verified this hypothesis.

Observations

  • Error occurred after extended use of the question tool without intervening user messages
  • Session reached 145,882 tokens (exceeds 128k limit)
  • Compaction failed even with stripMedia: true

Relevant Code Paths

  • compaction.ts:44-47 - isOverflow logic
  • compaction.ts:224-232 - error when compaction fails
  • prompt.ts:543-555 - pre-send overflow check
  • processor.ts:359-360 - error detection

Environment

  • Model: github-copilot/claude-opus-4.6 (128k context, 64k output)

Plugins

@franlol/opencode-md-table-formatter@0.0.3, @tarquinen/opencode-dcp@3.0.1, opencode-handoff@0.5.0; disabling opencode-dcp doesn't help

OpenCode version

1.2.25

Steps to reproduce

Steps to Reproduce

Create a minimal reproduction by configuring the model to continuously generate long responses:

  1. Create an AGENTS.md with instructions to never end responses and always continue using the question tool:
# Never-Ending Response Protocol

After every response, you MUST use the question tool to ask a follow-up question. Never provide a final answer. Always expand on your previous response with additional details, examples, and explanations before asking the next question.

This continues indefinitely until the session hits context limits.
  1. Start a session with a model with limited context (e.g., 128k)
  2. The model will continuously generate responses and accumulate context
  3. Eventually the session will exceed the context limit and trigger the compaction failure

Screenshot and/or share link

No response

Operating System

No response

Terminal

No response

Activity

  1. added
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    on Mar 13, 2026
  2. github-actions commented on Mar 13, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

  3. removed
    bugSomething isn't working
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    on May 3, 2026
  4. martin-braun commented on May 29, 2026

    @martin-braun
    Image

    same problem with GPT 5.5, using 400k of 1m, so I would assume it can compact itself, but nope. A bit annoying. Does this imply my agent files and loaded skills together fill the other 600k already? Or does it try to attach all the tokens from my subagents too? I was hoping it would not, or at least show me the sum of all tokens, because the value seems to be not accurate then ..

  5. added a commit that references this issue on Jul 15, 2026
    7adb9f1
  6. johncoffee715 commented on Jul 15, 2026

    @johncoffee715

    Proposed Fix: Estimate pending context in isOverflow()

    PR: https://github.com/johncoffee715/opencode/pull/1 (em branches )

    Problem

    uses which only reflects the previous step's API response. It doesn't account for upcoming context (system prompts, tool schemas, pending tool outputs), causing overflow detection to trigger too late.

    Solution

    Added parameter to in :

    Additional Fixes

    This PR also includes fixes for related overflow issues:

    Testing

    • All 52 existing compaction tests pass
    • No regressions detected

    Commits

    fa47663 fix(session): reserve full maxOutputTokens in usable() for limit.input models
    7adb9f1 fix(session): estimate pending context in isOverflow() and allow Continue after compaction overflow
    daad47d fix(session): check overflow after tool-result to prevent context overflow on next step
    
  7. EvolveAegis commented on Aug 1, 2026

    @EvolveAegis

    Hit the same loop. I think the silent bail is here, before Compaction.Started is published (file is under packages/core/ on current main, the V2 path):

    packages/core/src/session/compaction.ts

    // :184 — guard
    if (Token.estimate(summaryPrompt) > context - summaryOutput) return false
    // :186 — Started is published only AFTER the guard returns
    yield* dependencies.events.publish(SessionEvent.Compaction.Started, { ... })

    When the summary prompt itself is too big for the window, compactAfterOverflow returns false with no event, no log, and llm.stream is never called. There is no logger call anywhere in compaction.ts, so the failure isn't observable from the outside — the session just keeps re-requesting full history every turn.

    Why the guard is reachable, not shadowed by the auto-compact trigger:

    • Trigger (:230-234) compacts when estimate({system, messages, tools}) > context - max(output, buffer=20k) (DEFAULT_BUFFER :12).
    • Guard (:184) bails when estimate(summaryPrompt) > context - 4,096 (SUMMARY_OUTPUT_TOKENS :15).
    • summaryPrompt = instructions + template + head, and select() (:128-159) only carves out the recent ~8k (DEFAULT_KEEP_TOKENS :13) as recent. head is everything older, joined raw with no truncation fallback (buildPrompt :161-168).

    So head is unbounded. Once a session is past the compact threshold and grows a bit more without a successful compaction, summaryPrompt crosses context - 4,096 and the guard trips on every subsequent attempt.

    There's also a faster way in: packages/core/src/session/runner/llm.ts:282-288 passes compactAfterOverflow as recoverOverflow, so a single oversized turn that overflows the provider (e.g. pasting a large file) routes straight into this path — no gradual growth needed.

    Repro driving the real make().compactAfterOverflow (synthetic history), verified against sst/opencode@69a8066:

    git clone https://github.com/sst/opencode && cd opencode
    git checkout 69a8066
    bun install
    cd packages/core
    cat > zz-repro.test.ts <<'EOF'
    import { test, expect } from "bun:test"
    import { Effect, Stream } from "effect"
    import { Model } from "@opencode-ai/llm"
    import { make, buildPrompt } from "./src/session/compaction"
    import { Token } from "./src/util/token"
    
    const CONTEXT = 200_000
    const OUTPUT = 4_096
    
    function deps(c: { stream: number; events: string[] }) {
      return {
        config: [] as any,
        events: {
          publish: (ev: any) => { c.events.push(String(ev?.type ?? ev?.tag ?? ev)); return Effect.void },
        } as any,
        llm: { stream: () => { c.stream++; return Stream.empty as any } } as any,
      }
    }
    const hist = (n: number, chars: number) =>
      Array.from({ length: n }, (_, i) => ({
        seq: i,
        message: { type: "user", text: `m${i}-` + "x".repeat(chars) } as any,
      }))
    const model = Model.make({
      id: "t", provider: "t",
      route: { defaults: { limits: { context: CONTEXT, output: OUTPUT } } } as any,
    } as any)
    
    test("overflow-sized history -> silent false (synthetic)", async () => {
      const c = { stream: 0, events: [] as string[] }
      const r: any = await Effect.runPromise(
        make(deps(c) as any).compactAfterOverflow(
          { sessionID: "s", entries: hist(43, 20_000), model, request: {} as any }) as any)
      const prompt = buildPrompt({ context: ["h".repeat(860_000 - 32_000)] })
      console.log(`summaryPrompt est=${Token.estimate(prompt)} (threshold ${CONTEXT - OUTPUT}); returned ${r}; stream=${c.stream}; events=${JSON.stringify(c.events)}`)
      expect(r).toBe(false)
      expect(c.stream).toBe(0)
      expect(c.events).toHaveLength(0)
    })
    
    test("moderate history -> guard passes (synthetic)", async () => {
      const c = { stream: 0, events: [] as string[] }
      await Effect.runPromise(
        make(deps(c) as any).compactAfterOverflow(
          { sessionID: "s", entries: hist(8, 20_000), model, request: {} as any }) as any)
      expect(c.stream).toBe(1)            // summarization attempted
      expect(c.events[0]).toMatch(/started/i)
    })
    EOF
    bun test zz-repro.test.ts

    Observed (synthetic history, 43 messages × 20k chars ≈ 215k tokens):

    summaryPrompt est=207293 (threshold 195904); returned false; stream=0; events=[]
    2 pass
    

    Both tests pass. The contrast case (8 messages, ~40k tokens) clears the guard, publishes session.next.compaction.started, and calls llm.stream once — same function only attempts summarization while history is still small.

    The exact token numbers depend on context window, output reserve, and tokenizer; the relevant part is the branch: the largest sessions are the ones that get silently refused.

    Minimal fix idea: when the summary prompt exceeds the window, hard-truncate head to a fixed budget (no LLM pass) instead of returning false, and emit a warn event so the path is observable. Localized to :184.

    Adding as a comment since this issue already covers the symptom; happy to open a PR for the truncation fallback if it sounds right.

  8. Zuozhuo commented on Sep 14, 2026

    @Zuozhuo

    Description

    Description

    Sessions can trigger the error:

    "Session too large to compact - context exceeds model limit even after stripping media"

    In my case, using a model with a 128k context limit, the session grew to 145,882 tokens. Notably, there hadn't been a user message in a while - only responses through the question tool accumulating in the conversation.

    What I Think Is Happening

    The overflow detection in isOverflow() (packages/opencode/src/session/compaction.ts:32-48) appears to use token counts from the previous turn's completed API response (lastFinished.tokens). This may not account for content that will be added in the next API call:

    1. New user message
    2. System prompts
    3. Reminders
    4. Tool schemas

    This could create a timing gap where sessions exceed the safe threshold before compaction is triggered. However, I haven't fully verified this hypothesis.

    Observations

    • Error occurred after extended use of the question tool without intervening user messages
    • Session reached 145,882 tokens (exceeds 128k limit)
    • Compaction failed even with stripMedia: true

    Relevant Code Paths

    • compaction.ts:44-47 - isOverflow logic
    • compaction.ts:224-232 - error when compaction fails
    • prompt.ts:543-555 - pre-send overflow check
    • processor.ts:359-360 - error detection

    Environment

    • Model: github-copilot/claude-opus-4.6 (128k context, 64k output)

    Plugins

    @franlol/opencode-md-table-formatter@0.0.3, @tarquinen/opencode-dcp@3.0.1, opencode-handoff@0.5.0; disabling opencode-dcp doesn't help

    OpenCode version

    1.2.25

    Steps to reproduce

    Steps to Reproduce

    Create a minimal reproduction by configuring the model to continuously generate long responses:

    1. Create an AGENTS.md with instructions to never end responses and always continue using the question tool:

    Never-Ending Response Protocol

    After every response, you MUST use the question tool to ask a follow-up question. Never provide a final answer. Always expand on your previous response with additional details, examples, and explanations before asking the next question.

    This continues indefinitely until the session hits context limits.
    2. Start a session with a model with limited context (e.g., 128k)
    3. The model will continuously generate responses and accumulate context
    4. Eventually the session will exceed the context limit and trigger the compaction failure

    Screenshot and/or share link

    No response

    Operating System

    No response

    Terminal

    No response

    I can reproduce what appears to be the same issue with a local vLLM backend.

    Environment

    • Model: Qwen3.8-27B
    • Backend: local vLLM
    • Provider: OpenAI-compatible
    • vLLM context window: 256K
    • OpenCode auto-compaction: enabled

    Observed behavior

    The session reached approximately:

    241.8K tokens (94%)
    

    OpenCode then automatically triggered compaction, but the compaction failed almost immediately with:

    Session too large to compact - context exceeds model limit even after stripping media
    

    The UI shows:

    Compaction · Qwen3.8-27B
    241.8K (94%)
    

    So in this case auto-compaction is being triggered correctly, but it appears to trigger too late: by the time compaction starts, the compaction request itself no longer fits into the model's 256K context window.

    This is running against a local vLLM endpoint, so this also reproduces outside the original GitHub Copilot / Claude setup reported here.

    It looks like the effective size of the compaction request may be larger than the token count displayed in the TUI, possibly because of additional system prompts, tool schemas, compaction instructions, or output-token reservation.

    Expected behavior would be for OpenCode to trigger compaction with enough headroom that the compaction request itself can still fit inside the model context window.

    I can provide my OpenCode provider configuration and vLLM launch command if useful.

    Image
  9. thdxr commented on Oct 11, 2026

    @thdxr
    Member

    Thanks for the report! OpenCode 2.0 is now out and development is focused on V2, so we're closing issues filed against OpenCode 1. Please upgrade (opencode upgrade --major, details: https://anoma.ly/notes/opencode-2-0), and if this is still relevant on 2.0, reopen this issue or open a new one with your 2.0 version.

    — from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions