[Bug]: Session processor retries indefinitely with unbounded exponential backoff — no max retries or circuit breaker

Description

When the upstream LLM provider (in my case GitHub Copilot / api.githubcopilot.com) returns transient errors, the session processor enters an infinite retry loop with exponential backoff that grows unbounded. There is no maximum retry count, no circuit breaker, and no configurable limit. Sessions become effectively dead for hours.

Root Cause Analysis

In packages/opencode/src/session/processor.ts, the process() method has a while (true) loop. When a retryable error occurs in the catch block:

const retry = SessionRetry.retryable(error)
if (retry !== undefined) {
    attempt++
    const delay = SessionRetry.delay(attempt, ...)
    SessionStatus.set(input.sessionID, {
        type: "retry",
        attempt,
        message: retry,
        next: Date.now() + delay,
    })
    await SessionRetry.sleep(delay, input.abort).catch(() => {})
    continue  // ← loops forever
}

The attempt counter increments without limit, and in retry.ts, the delay function when response headers ARE present has no cap:

// WITH headers → no max delay cap!
return RETRY_INITIAL_DELAY * Math.pow(RETRY_BACKOFF_FACTOR, attempt - 1)
// 2000 * 2^(attempt-1) → attempt 10 = 1,024,000ms (17 min), attempt 15 = 32,768,000ms (9 hrs)

Only the NO-headers path has a 30-second cap (RETRY_MAX_DELAY_NO_HEADERS). When the upstream sends error responses WITH headers (as GitHub Copilot does), the backoff grows without bound.

Evidence

From my logs (~/.local/share/opencode/log/):

173 consecutive retry failures over 2.5 hours (16:46–19:27 UTC on 2026-03-15)
Provider: github-copilot, model: claude-opus-4.6, endpoint: api.githubcopilot.com/chat/completions
Error: AI_APICallError: Could not relay message upstream
Backoff grew from ~2s (attempt feat: compact and other improvements #1) to 7+ minutes (attempt diff rendering issues #10) and continued growing
Session was completely unresponsive — had to manually kill and restart

Related Issues

Endless loop of TimeoutOverflowWarning when exceeding Github Copilot monthly message limit #6430 — Fixed the TimeoutOverflowWarning when delay exceeds 32-bit int, but did NOT add a max retry count
[Bug]: Model rate limit error causes immediate termination instead of auto-retry #16994 — Rate limit errors not being detected as retryable (opposite problem — errors that SHOULD retry don't)

Expected Behavior

Maximum retry count — After N retries (configurable, default ~10), stop retrying and surface the error to the user
Maximum backoff cap — Even with headers, cap at e.g. 60 seconds (the 30s RETRY_MAX_DELAY_NO_HEADERS should apply universally)
Circuit breaker — After repeated failures, stop attempting for a longer period and notify the user rather than silently blocking
Configurable — Allow users to set maxRetries and maxBackoffMs in opencode.json provider config

Proposed Fix

In retry.ts:

export const RETRY_MAX_ATTEMPTS = 10  // NEW: stop after 10 retries
export const RETRY_MAX_DELAY_WITH_HEADERS = 60_000  // NEW: cap at 60s even with headers

In processor.ts, inside the catch block:

if (retry !== undefined) {
    attempt++
    if (attempt > SessionRetry.RETRY_MAX_ATTEMPTS) {
        // Circuit breaker: stop retrying, surface error
        input.assistantMessage.error = error
        Bus.publish(Session.Event.Error, { sessionID: input.sessionID, error })
        SessionStatus.set(input.sessionID, { type: "idle" })
        break  // ← EXIT the while(true) loop
    }
    const delay = SessionRetry.delay(attempt, ...)
    // ...
}

In retry.ts delay function, apply cap universally:

return Math.min(
    RETRY_INITIAL_DELAY * Math.pow(RETRY_BACKOFF_FACTOR, attempt - 1),
    RETRY_MAX_DELAY_WITH_HEADERS  // ← always cap
)

Environment

OpenCode: Latest (installed via curl -fsSL https://opencode.ai/install | bash)
OS: macOS Sequoia 15.x
Terminal: iTerm2 / tmux
Provider: GitHub Copilot (github-copilot)

This issue might be a duplicate of existing issues. Please check:

Infinite retry loop when StreamIdleTimeoutError occurs during tool input generation #12234: Same root cause — infinite retry loop in session/processor.ts with unbounded exponential backoff via retry.ts, specifically triggered by retryable errors with no max attempt count
Subagent enters infinite retry loop on edit/write tool failure, causing excessive API costs #17169: Same request for a circuit breaker / max retry limit to prevent infinite retry loops consuming excessive API usage

Please review those issues and consider whether this is the same underlying problem before proceeding.

This isn't a duplicate — it's a specific, reproducible failure mode in the retry logic.

The root cause is two compounding issues in packages/opencode/src/session/:

retry.ts line 54: When response headers exist but don't include retry-after, the exponential backoff has no upper bound. The no-headers path caps at 30s, but the with-headers fallback just returns 2000 * 2^(attempt-1) unbounded.
processor.ts ~line 370: The while(true) loop has no max attempt check — it increments attempt, sleeps the unbounded delay, and continues forever.

I reproduced this today: 11 consecutive AI_APICallError: Could not relay message upstream with delays escalating to 202s between attempts, 542 seconds total before I killed it manually. Process was burning 10%+ CPU the whole time.

Related issues #12234 and #17169 describe similar symptoms (runaway costs, stuck sessions) but from different entry points. The retry loop is the common factor.

I've submitted a fix PR: #17668 — adds a 10-attempt circuit breaker and 60s backoff cap for the with-headers path.

Quick update: built opencode from source with the fix from PR #17668 and ran it against the two stuck projects from my original report. Both moved forward successfully — no more infinite retry loops.

The patched version correctly caps retries at 10 attempts and breaks out of the loop with a clean error state, instead of spinning indefinitely with exponential backoff.

Confirming on 1.15.6, OpenRouter HTTP 500, identical backoff curve to closed-as-duplicate #26675

Hit this on opencode run --agent plan (headless) with moonshotai/kimi-k2-thinking via OpenRouter. First chat-completion request after the initial bash tool call returned HTTP 500 with body {"error":{"message":"Internal Server Error","code":500}}. isRetryable: true classified it, and the retry policy ran for the full 30-minute wrapper budget before SIGTERM.

Observed backoff (from ~/.local/share/opencode/log/<run>.log, 10 ERROR service=llm entries on the same session.id):

Attempt	Timestamp (UTC)	Delay from prior
1	22:30:28	, (initial)
2	22:30:30	2s
3	22:30:34	4s
4	22:30:42	8s
5	22:30:58	16s
6	22:31:30	32s
7	22:32:35	65s
8	22:34:43	128s
9	22:39:00	257s
10	22:47:32	512s

Exactly matches the curve described in #26675 (which got auto-closed for title format, not because the bug was invalid). Same RETRY_INITIAL_DELAY=2000ms, RETRY_BACKOFF_FACTOR=2, no max attempt cap.

Confirms the bug is provider-agnostic, #26675 saw it with DeepSeek-V4-Pro direct, #18684 with Anthropic context-overflow, this run with OpenRouter routing moonshotai/kimi-k2-thinking. Any isRetryable: true 5xx triggers it. User-side wrapper timeout is the only escape on opencode 1.15.6.

Config used (~/.config/opencode/opencode.json):

"openrouter": {
  "npm": "@ai-sdk/openai-compatible",
  "options": {
    "apiKey": "{env:OPENROUTER_API_KEY}",
    "baseURL": "https://openrouter.ai/api/v1"
  },
  "models": {
    "moonshotai/kimi-k2-thinking": { "limit": { "context": 262144, "output": 32768 } }
  }
}

Worth consolidating with #21960 / #25733 so the assignees aren't duplicating analysis? Happy to attach the full log if useful.

Drafted by: Claude Code (Opus 4.7) via local repro
Sources reviewed: local ~/.local/share/opencode/log/2026-05-22T223017.log (opencode 1.15.6 headless run), OpenRouter /credits API, GitHub issues #26675 / #18684 / #21960 / #25733 / #27361
Peer review: none yet
Caveats: did not capture the full request body; the 500 may have been triggered by prompt_cache_key being forwarded to a non-Moonshot OpenRouter downstream provider, but the retry-storm behavior is the bug regardless of trigger
Please verify before sending.

PR #30040 adds retry cap export + test coverage.

Is it still waiting for a fix?

Ideally there should be a hotkey to "retry now"

Confirming on opencode 1.3.3 (Sep 26, 2026) — controlled repro, production gap math, and a headless-visibility angle that hasn't been covered above.

Controlled repro — local mock provider returning 500 with only content-type in the response headers (no retry-after), body mock upstream error:

Observed inter-attempt deltas: 2.03s / 4.01s / 8.01s / 16.02s / 32.02s / 64.02s — a perfect x2 ladder from RETRY_INITIAL_DELAY * 2^(attempt-1) with no cap applied, and the process emits zero output for the entire ladder.

Production gap quantization (fingerprint for anyone triaging mystery "hangs"): multi-hour silent stalls observed in production run logs land on exact ladder sums. During one provider outage window we recorded silent gaps of ~4.55h / 9.1h / 18.2h / 36.4h; e.g. the 36.4h case is 131,084.8s vs the ladder sum Σ(2s .. 2^16s) = 131,070s plus ~1s/attempt overhead across 15 attempts. If an automated run looks "hung" and the silence is a power-of-two multiple of ~2 seconds, it is this ladder, not a deadlock — worth checking before assuming a transport/CLI bug.

Headless impact (distinct from the UI-stuck reports above): opencode run --format json emits nothing during retries — SessionStatus.set({type:"retry"}) is internal-only and no retry-status part is emitted on the run stream. For programmatic consumers (CI, agent harnesses), a run inside the ladder is indistinguishable from a dead process. Also note the SDK layer is called with retries ?? 0, so all retrying funnels into this session loop — there is no second safety net. A retry-status event on the run stream would make this whole failure class observable.

Fix scope: #33733 is the right first step — the 30s RETRY_MAX_DELAY_NO_HEADERS cap is effectively dead in production because real 5xx responses almost always carry headers. But even with the delay capped, the processor loop keeps its unbounded attempt count, so a persistent outage retries forever at the cap and the run never surfaces a terminal error. End-to-end close = delay cap (#33733) + max-attempts bound + a retry event on the run stream.

Closing the unbounded retry-count behavior reported here. The shipped version boundaries are:

OpenCode V1: v1.18.17+ — fix(opencode): cap session retries with jitter #41939 limits session-level retries to five; v1.18.16 did not have that limit.
OpenCode V2: v2.0.0+ — the session runner already had a bounded schedule (four retries). v2.0.6+ (fix(core): retry provider failures for about 84s with 10s gap cap #49441) allows ten retries but caps the ordinary exponential gap at about 10 seconds. V2 also bounds provider-requested Retry-After delays to 15 minutes (fix(core): cap provider retry-after delays at fifteen minutes #45787).

This is specifically the fix for retrying forever, not a claim that every requested backoff/visibility change shipped. In V1, the fallback for a response with headers but no valid retry-after still does not apply the 30-second per-gap cap (including in v1.18.32); #33733 remains open for that. A large explicit Retry-After can still cause a long, finite wait. These changes also do not establish that headless run --format json emits retry progress.

The Sep 26 reproduction reports v1.3.3, which predates the V1 fix. If indefinite retries still occur on v1.18.17+ or V2, please provide the exact version and a bounded repro so we can investigate that separately.

Navigation Menu

Repository navigation

[Bug]: Session processor retries indefinitely with unbounded exponential backoff — no max retries or circuit breaker #17648

Description

Root Cause Analysis

Evidence

Related Issues

Expected Behavior

Proposed Fix

Environment

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Bug]: Session processor retries indefinitely with unbounded exponential backoff — no max retries or circuit breaker #17648

Description

Description

Root Cause Analysis

Evidence

Related Issues

Expected Behavior

Proposed Fix

Environment

Activity

github-actions commented on Mar 15, 2026

dawidbednarczyk commented on Mar 15, 2026

dawidbednarczyk commented on Mar 15, 2026

hampsterx commented on May 23, 2026

nikhilkulkarni1755 commented on Jun 2, 2026

itamargnetapp commented on Jul 20, 2026

gajus commented on Sep 11, 2026

skillsforsuccess commented on Sep 26, 2026

opencode-agent commented on Sep 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions