max_tokens defaults to 32000 when using a custom provider

Hi,

I'm using LLM (Anthropic, OpenAI and Bedrock) models through an OpenAI-compliant LLM API Gateway. I configured a custom provider in opencode to use this LLM Gateway using the opencode.json file, e.g.

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "myprovider": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Custom LLM Gateway",
      "options": {
        "baseURL": "<GATEWAY_URL",
        "apiKey": "<API_KEY>",
      },
      "models": {
        "openai:gpt-4o-mini": {
          "name": "gpt-4o-mini"
        }
        "anthropic:claude-3-5-haiku-20241022": {
          "name": "claude-3-5-haiku-20241022"
        }
      }
    }
  }
}

When I try to use either of these models through opencode, I'm receiving error messages that suggest that something (maybe opencode, maybe the Vercel AI SDK) is defaulting the max_tokens OpenAI Completions field to 32000, which unfortunately is far too large for these models.

As an example, here is the error I see in the LLM Gateway logs when attempting to use the gpt-4o model (I see a similar error with Anthropic models):

  "error": {
    "message": "max_tokens is too large: 32000. This model supports at most 16384 completion tokens, whereas you provided 32000.",
    "type": "invalid_request_error",
    "param": "max_tokens",
    "code": "invalid_value"
  }
}

I've tried searching the opencode and Vercel AI SDK codebases to try to find where this max_tokens value is being set, but I unfortunately can't find it. Assuming it is being set by opencode, it would be ideal if this value were left unset when using custom providers, as in the case of my LLM Gateway, it already keeps track of the max_token parameter for each provider / model.

A couple other notes:

The reason I know the max_tokens parameter is being set by opencode or Vercel SDK, is because when I write my own http request to my LLM Gateway without specifying max_tokens, the query goes through, e.g.

curl --header 'Authorization: Bearer <API TOKEN>' -H "Content-Type: application/json" -X POST --data '{"model":"openai:gpt-4o-mini","messages":[ {"role": "user", "content": "Hello!"}]}' <GATEWAY_URL>/v1/chat/completions

When I modify the opencode.json file to include context and max output tokens, opencode works as expected (I know this is a workaround, but really trying not to have to hardcode context and output tokens in both places):

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "myprovider": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Custom LLM Gateway",
      "options": {
        "baseURL": "<GATEWAY_URL",
        "apiKey": "<API_KEY>",
      },
      "models": {
        "openai:gpt-4o-mini": {
          "name": "gpt-4o-mini",
          "limit": {
            "context": 10000,
            "output": 5000
          }
        }
        "anthropic:claude-3-5-haiku-20241022": {
          "name": "claude-3-5-haiku-20241022",
          "limit": {
            "context": 10000,
            "output": 5000
          }
        }
      }
    }
  }
}

Please let me know if my understanding is wrong, and this parameter is being set elsewhere. Also, please let me know if there is any additional information required to troubleshoot.

Thank you, and thanks for making such an awesome tool!

opencode needs to know what the max output length is - when it doesn't it uses 32_000

i can't think of a way around this - you need to specify it somewhere

try this please. Worked for me with kimi k2 free.

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "openrouter": {
      "models": {
        "moonshotai/kimi-k2:free": {
          "options": {
            "provider": {
              "order": ["baseten", "together", "openrouter"],
              "allow_fallbacks": true
            },
            "transform": "middle-out",
            "max_tokens": 6000
          }
        }
      }
    }
  }
}

I'm having issues with this as well :( I've tried throwing the max_token setting everywhere I could think, but it still shows up when I look at the logs in LM studio as

25-12-04 17:11:17 [DEBUG]
 Received request: POST to /v1/chat/completions with body  {
  "model": "qwen/qwen3-coder-30b",
  "max_tokens": 32000,
  "temperature": 0.55,

The config

{
    "$schema": "https://opencode.ai/config.json",
    "permission": {
      "edit": "allow",
      "bash": "allow",
      "webfetch": "ask"
    },
    "provider": {
      "lmstudio": {
        "npm": "@ai-sdk/openai-compatible",
        "name": "Mac Mini",
        "options": {
          "baseURL": "http://addres:8123/v1",
          "timeout": 100000000000,
	  "max_tokens": 200000,
          "max_completion_tokens": 200000
        },
        "models": {
          "qwen/qwen3-coder-30b": {
            "name": "Qwen Coder (local)",
            "max_tokens": 131072,
            "max_completion_tokens": 200000,
            "limit": {
              "context": 209600,
              "output": 50000
            }
          }
        }
      }
    }
}

It can be modified by setting the OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX environment variable.

opencode needs to know what the max output length is - when it doesn't it uses 32_000i can't think of a way around this - you need to specify it somewhere

If the server can produce an error saying that the max tokens are X, then opencode could reasonably be expected to detect that response and reduce its max tokens to X, activating a context compaction if necessary.

I ended up doing:

"limit": { "context": 0, "output": 8196 }

Perhaps some playing around and the error should subside

@njbrake

@njbrake did you manage to solve the issue, I'm facing a similar issue with KoboldCpp endpoint on Openai compatibility mode. It always truncates the response at 640 tokens.
I tried the limit (context and output) settings but they don't seem to take any effect.

"limit": { "context": 0, "output": 8196 }

Thank. Better to use 8192 though? Can confirm it is working. Prefer to put in opencode.json next to model than in global environment variable.

opencode needs to know what the max output length is - when it doesn't it uses 32_000i can't think of a way around this - you need to specify it somewhere

I think this information is returned by /v1/models

"limit": { "context": 0, "output": 8196 }

Thank. Better to use 8192 though? Can confirm it is working. Prefer to put in opencode.json next to model than in global environment variable.

Yeah I change it to anything to be honest haha, just wanted it to work,

To stay organized issues are automatically closed after 90 days of no activity. If the issue is still relevant please open a new one.

OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX

opencode/packages/opencode/src/provider/transform.ts

Line 21 in 58a9991

    
           export const OUTPUT_TOKEN_MAX = Flag.OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX || 32_000

opencode/packages/opencode/src/flag/flag.ts

Line 69 in 58a9991

    
           export const OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX = number("OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX")

opencode/packages/opencode/src/flag/flag.ts

Lines 87 to 92 in 58a9991

    
           function number(key: string) { 
        
             const value = process.env[key] 
        
             if (!value) return undefined 
        
             const parsed = Number(value) 
        
             return Number.isInteger(parsed) && parsed > 0 ? parsed : undefined 
        
           }

OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX

opencode/packages/opencode/src/provider/transform.ts

Line 21 in 58a9991

export const OUTPUT_TOKEN_MAX = Flag.OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX || 32_000
opencode/packages/opencode/src/flag/flag.ts

Line 69 in 58a9991

export const OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX = number("OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX")
opencode/packages/opencode/src/flag/flag.ts

Lines 87 to 92 in 58a9991

function number(key: string) {
const value = process.env[key]
if (!value) return undefined
const parsed = Number(value)
return Number.isInteger(parsed) && parsed > 0 ? parsed : undefined
}

OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX is worked, Thx a lot.

This bug still doesn't seem to be fixed unfortunately. you just have to use the env var

Same here. This bug still exist... After export OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX=1048576, the opencode still does not work as expected.
"local-proxy": {
"npm": "@ai-sdk/openai-compatible",
"name": "A Local Proxy",
"options": {
"baseURL": "http://localhost:8081/v1"
},
"max_tokens": 1048576,
"models": {
"xxxx": {
"name": "xxx",
"tool_call": true,
"limit": {
"context": 262144,
"input": 262144,
"output": 262144
},
"options": {
"tools": true
}
},

When I looked at the raw response
{
"model": "xxx",
"max_tokens": 131072, <-- this part still changed for some unknown reasons.
"tools": [
{
"type": "function",
"function": {
"name": "ast_grep_replace",
"description": "Rewrite code by AST pattern (25 languages). Dry-run by default.\nBoth pattern and rewrite use AST syntax ($VAR for one node, $$$ for many) - regex does NOT work.\nMeta-variables captured in pattern can be reused in rewrite to preserve matched content.\nExample: pattern="console.log($MSG)" rewrite="logger.info($MSG)"\nFor text-only replacement or regex features, use a text editor instead.",

the bug still exists

Bug still exist. Its happening to me with openrouter deepseek-v4-pro free. I tried with this configuration in the

opencode.jsonc
  "provider": {
      "models": {
        "openrouter/deepseek/deepseek-v4-pro": {
          "options": {
            "max_tokens": 12400
          }
        }
      }
  },

But the problem still happens, only way to make it work is with the env varable OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX

Bug still exists

Same issue when using : ollama launch opencode --model glm-5.3-flash:cloud

Navigation Menu

max_tokens defaults to 32000 when using a custom provider #1735

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	function number(key: string) {
	const value = process.env[key]
	if (!value) return undefined
	const parsed = Number(value)
	return Number.isInteger(parsed) && parsed > 0 ? parsed : undefined
	}

max_tokens defaults to 32000 when using a custom provider #1735

Description

Activity

thdxr commented on Aug 9, 2025

patrickwork28 commented on Sep 7, 2025

njbrake commented on Dec 4, 2025

jimlaren1 commented on Jan 14, 2026

lee-b commented on Jan 20, 2026

Onik97 commented on Jan 27, 2026

mercurial-moon commented on Jan 29, 2026

toralux commented on Feb 7, 2026

lee-b commented on Feb 7, 2026

Onik97 commented on Feb 7, 2026

github-actions commented on Apr 9, 2026

CoryMConway commented on Apr 9, 2026

pollyduan commented on Apr 29, 2026

Xynonners commented on May 11, 2026

abysslover commented on May 12, 2026

xantilon commented on Jun 11, 2026

alhazacod commented on Jun 15, 2026

awaescher commented on Sep 16, 2026

benjamin-bonneton commented on Sep 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions