Skip to content

API Rate Limit Not Properly Handled - Model Stops Responding After Few Queries #11104

Closed

Description

@xgc8008-zed

I'm experiencing an issue when using NVIDIA model APIs where after asking a few questions, the model suddenly stops responding with no output. However, the animation in the bottom-left
corner continues running, indicating the system is still attempting to process. This can persist for several hours.

 **Environment:**
 - API Provider: https://build.nvidia.com/
 - Rate Limit: 40 rpm
 - Models Affected: GLM-4.7 or other models from NVIDIA API catalog                                                                                                                             

 **Steps to Reproduce:**
 1. Start a conversation with the model (e.g., GLM-4.7 or other models)
 2. Ask 2-3 questions successfully                                                                                                                                                                
 3. On subsequent questions, the model stops responding                                                                                                                                           
 4. Animation continues running indefinitely                                                                                                                                                      
 5. All submitted tasks remain stuck in the task list
                                                                                                                                                                                                  
 **Expected Behavior:**
 - Model should respond to all queries within rate limits
 - If rate limit is exceeded, user should receive clear error message
 - System should handle rate limit gracefully

 **Actual Behavior:**
 - Model silently fails after 2-3 queries                                                                                                                                                         
 - Animation continues but no response is ever returned                                                                                                                                           
 - Issue persists for several hours                                                                                                                                                               
 - No error message displayed to indicate rate limit was reached
                                                                                                                                                                                                  
 **Analysis:**                                                                                                                                                                                    
 It appears the API returns HTTP 429 (Too Many Requests) or similar rate limit error when usage exceeds 40 rpm, but the client (opencode) does not properly detect or display this error,         
 leaving the user in a confused state where the UI shows activity but no actual response is generated.

 **Request:**
 Please implement proper rate limit error handling that:
 1. Detects rate limit responses from the API
 2. Displays a clear error message to the user                                                                                                                                                  
 3. Shows remaining quota or wait time   
 4. Prevents indefinite hanging states   
 5. Provides proper timeout behavior instead of leaving tasks stuck

Activity

  1. github-actions commented on Jan 29, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    Your issue specifically focuses on silent failures and missing error messages for NVIDIA API rate limits (HTTP 429). While these related issues address rate limiting broadly, they may provide context or overlapping solutions.

    Feel free to ignore if none of these address your specific case.

  2. github-actions commented on Apr 11, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 90 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions