Skip to content

[FEATURE]:Auto-Failover for Rate Limit Resilience #42513

Closed as not planned
Closed as not planned

Description

@my-txz

Been hitting this nonstop on longer refactoring and large codebase work — OpenCode only takes one API key per provider right now, and the second that key hits a 429 rate limit, runs into a random 500 server error, or burns through daily quota, the whole agent session dies halfway through. You have to manually swap keys and restart everything, which completely defeats the point of letting it run unattended.

I know issue #7602 covers model-level fallback, but this is a separate, narrower request. That one is about falling back to different model types when a provider fails; this is about pooling multiple keys for the same provider and same model to spread out request load and fail over instantly when a single key gets throttled. No manual intervention, no lost progress. They complement each other, not overlap.

The core ask is simple: build key pooling natively into the LLM adapter layer. Let users stack keys via numbered environment variables, like OPENAI_KEY_1, OPENAI_KEY_2, and cycle through them round-robin by default to even out request pressure. If a request comes back 429, immediately retry that exact same call on the next available key instead of throwing an error and killing the task. Honor the Retry-After header when the provider sends it, otherwise stick the throttled key into a 60-second cooldown before putting it back in rotation.

Error handling can stay straightforward, no need for anything overengineered. 5xx server errors trigger a short cooldown after a couple consecutive failures so it doesn't hammer a broken endpoint. 401/403 auth failures mark the key as invalid entirely, stop routing traffic to it, and show a clear notice in the UI. Keys that hit daily quota limits stay disabled until UTC midnight reset.

No breaking changes either. If you only have one key configured, everything behaves exactly like it does today. Would also be nice to tuck a tiny status indicator in the TUI footer — show the active key masked to its last 4 characters, and how many are sitting in cooldown — just so you can tell what's going on at a glance.

I'm keeping this intentionally narrow, focused solely on key pooling and failover logic. All the other ideas I brought up earlier around caching and repo structure I'll file as separate focused issues later. Just want to fix the rate-limit crash problem first, since it's the most consistent day-to-day frustration.

Activity

  1. github-actions commented on Aug 14, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    Please consider consolidating into #42502 or one of the existing threads rather than filing a new issue.

  2. my-txz commented on Aug 14, 2026

    @my-txz
    Author

    done

  3. github-actions commented on Oct 15, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions