I can not configure the limit for litellm #2787

Activity

1. github-actions commented on Sep 26, 2025

   

   github-actionsBot

   on Sep 26, 2025 – with GitHub Actions

   Contributor

   More actions

   This issue might be a duplicate of existing issues. Please check:

   • AI_APICallError with Qwen 3 Coder 480b: Please reduce the length of the messages or completion. Current length is 69553 while limit is 65536 #1551: Similar issue with Qwen 3 Coder 480b where context limits are not properly respected, causing "Please reduce the length of the messages or completion" errors
   • max_tokens defaults to 32000 when using a custom provider #1735: Reports that max_tokens defaults to 32000 when using custom providers with openai-compatible, and discusses how limit configuration in config.json should work as a workaround
   • Model options are not forwarded for openai-compatible providers unless provider.name option is given #971: Model options are not forwarded for openai-compatible providers unless provider.name option is given, which may affect limit configurations

   Feel free to ignore if none of these address your specific case.
2. rekram1-node commented on Sep 26, 2025

   

   rekram1-node

   on Sep 26, 2025

   Last edited by rekram1-node

   Collaborator

   More actions

   @m-da-costa put:

   includeUsage: true

   in the provider options

   ex:

    "litellm": {
         "npm": "@ai-sdk/openai-compatible",
         "options": {
           "baseURL": "https://my.domain.com/v1",
           "includeUsage": true
         },
         "models": {
           "cerebras/qwen-3-coder-480b": {
             "tools": true,
             "reasoning": true,
             "context_window": 120000,
             "limit": {
               "context": 120000,
               "output": 64000
             }
           },
         }
       }
   

   ❤️React with ❤️1Reacted by m-da-costa
3. rekram1-node

   self-assigned this

   on Sep 26, 2025
4. rekram1-node

   closed this as completedin #2788on Sep 26, 2025
5. rekram1-node commented on Sep 26, 2025

   

   rekram1-node

   on Sep 26, 2025

   Collaborator

   More actions

   going forward we will set this automatically but until next release you should be able to follow waht I mentioned above^^

   ❤️React with ❤️1Reacted by m-da-costa
6. m-da-costa commented on Sep 26, 2025

   

   m-da-costa

   on Sep 26, 2025

   Author

   More actions

   Ty so much @rekram1-node