Trailing-assistant 400 on llama.cpp/vLLM with thinking-on templates (Qwen3, DeepSeek-R1, GLM-thinking, Kimi-K2-Thinking, MiniMax-M2)

Symptom

Local OpenAI-compatible servers running thinking-on-by-default chat templates (llama.cpp --reasoning on, vLLM with reasoning, TGI with thinking, mistral.rs, etc.) reject any opencode request whose last message is role:"assistant", with:

HTTP 400 {"error":{"message":"Assistant response prefill is incompatible with enable_thinking."}}

opencode emits a trailing-assistant message in two situations, both of which trip this error:

Empty trailing assistant — message-v2.toModelMessagesEffect sometimes builds an assistant UIMessage whose only parts are [step-start, reasoning("")]. convertToModelMessages collapses that to content:"", which is sent as a trailing assistant turn.
Non-empty trailing assistant — session/prompt.ts deliberately injects a MAX_STEPS wrap-up instruction as role:"assistant" (response continuation / prefill).

Reproduction

Run a llama-server with a thinking template, e.g.:

llama-server --model Qwen3.5-9B-...gguf --reasoning on --jinja --port 8080

Point opencode at it via an @ai-sdk/openai-compatible provider in opencode.json.
Run any agent with a steps limit small enough to trigger MAX_STEPS, or any flow that emits an empty-reasoning assistant turn.
Observe HTTP 400s in the llama-server log.

Affected model families

Every 2025-2026 open-weight thinking family using enable_thinking-branching templates:

Qwen3 hybrid (all sizes), Qwen3-Thinking-2507, Qwen3-VL, Qwen3.5, Qwen3.6, QwQ-32B
DeepSeek-R1, R1-0528, V4 (when thinking on)
GLM-4.6, GLM-4.7 thinking
Kimi-K2-Thinking
MiniMax-M2

Not affected: Qwen2.5, Qwen3-Coder, Qwen3-Instruct-2507, all Anthropic/OpenAI/Google/Bedrock-Anthropic models (these either don't use enable_thinking branching or accept prefill natively).

Upstream / cross-framework references

Misc. bug: Qwen3.5 with Opencode: Assistant response prefill is incompatible with enable_thinking. ggml-org/llama.cpp#20861 — exact symptom (opencode + Qwen3.5), still open
server: allow assistant prefill when the prefill itself closes the <think> block ggml-org/llama.cpp#21889 — proposal to relax the check (open, no PR)
Skills cause "Assistant response prefill is incompatible with enable_thinking" with Qwen 3.5 mastra-ai/mastra#15234 — same symptom in a different framework

Why fix it in opencode

A llama.cpp-side fix is unlikely soon and would only cover llama.cpp. opencode is the boundary where the per-provider request shape is decided, and where capability data already lives. Fixing it here also covers vLLM/TGI/mistral.rs, which have analogous behaviour but no shared upstream change to wait for.

Proposed fix (three PRs)

Empty-trailing case — extend the existing transform.ts empty-content filter (currently anthropic + bedrock only) to @ai-sdk/openai-compatible. Refactors two near-identical map+filter chains into one helper.
Model.prefill capability — add prefill: optional Boolean on Model and on the user-facing config schema. No consumer wiring yet.
Consumer + runtime probe — ProviderTransform.canAcceptTrailingAssistant(model) with three-layer precedence (explicit / auto-inference / default true). session/prompt.ts MAX_STEPS routes between role:assistant and role:user based on it. Runtime probe of <baseURL>/props (llama.cpp) detects enable_thinking-branching templates automatically — no user config needed for the common case.

Thinking stays enabled in the request body throughout — only the role of the synthetic MAX_STEPS message changes from assistant to user. The model thinks and writes its wrap-up normally.

This issue might be a duplicate of existing issues. Please check:

[FEATURE]: Strip trailing role=assistant && content=="" messages before request #26320: Feature request to strip trailing role=assistant && content=="" messages before requests — addresses the empty-trailing-assistant case you describe
Azure AI Claude rejects requests after tool abort: assistant message prefill not supported #19517: Azure AI Claude rejects requests after tool abort — same root cause (trailing role:assistant rejected by non-native SDK paths), same two code paths (message-v2.ts error filtering and prompt.ts isLastStep prefill injection)
Claude models behind OpenAI-compatible proxies get 400 prefill errors #20415: Claude models behind OpenAI-compatible proxies get 400 prefill errors — same prefill error, proposes isClaudeModel() helper fix across transform.ts
Bug: OpenCode prompt loop continues after finish=stop, triggering prefill error on claude-opus-4-6 #17982: OpenCode prompt loop continues after finish=stop, triggering prefill error — same prefill error class, same prompt.ts injection as root cause
This model does not support assistant message prefill / Github Copilot with Opus 4.6 #13768: "This model does not support assistant message prefill" — broad tracking issue for prefill errors across providers (65 comments)

Your issue focuses specifically on thinking-on templates (llama.cpp/vLLM enable_thinking) and proposes a runtime probe of <baseURL>/props for auto-detection — that angle appears new. The underlying fix areas (empty-trailing filter in transform.ts, isLastStep prefill in prompt.ts) overlap significantly with the issues above. It may be worth commenting on one of those threads, particularly #19517 which has the most detailed matching root cause analysis.

Thanks. Mapped against the 5 cited issues:

Issue	Overlap with this report	Coverage by PR #27914 / #27915 / #27916
#26320	identical empty-trailing-assistant case	full (PR1)
#20415	same symptom via Claude-on-openai-compat	full (PR1, different approach: shared `filterEmptyContent` helper instead of `isClaudeModel()` extraction)
#13768	broad prefill tracker, mixed cases	partial (PR1 covers empty cases; PR2 schema enables `prefill:false` per-model)
#19517	non-empty trailing from aborted tool-call	partial (requires explicit `prefill:false` for Azure-Claude; out of probe scope)
#17982	native-Anthropic prompt-loop after `finish=stop`	not addressed (orthogonal: loop-gating, not transform-layer)

Distinct contributions of this report:

Failure class. All 5 cited issues are Claude-specific (@ai-sdk/anthropic or Claude-via-proxy). This report targets thinking-on chat templates (enable_thinking in Qwen3 / DeepSeek-R1 / GLM-thinking / Kimi-K2-Thinking / MiniMax-M2 on llama.cpp / vLLM / TGI). Different server, different rejection path, same surface error string.
Detection mechanism. Runtime probe of <baseURL>/props parsing chat_template for enable_thinking — auto-detects without user config or models.dev entries. Not proposed in any of the cited issues.
Empirical validation. 8/8 captured 400-bodies from the original llama.cpp log eliminated by PR1 + PR2 + PR3. Beyond error elimination: thinking-capable models (Qwen3-Thinking, DeepSeek-R1, GLM-thinking, Kimi-K2-Thinking, MiniMax-M2) now complete their reasoning traces uninterrupted — the previous prefill-rejection truncated mid-thought and aborted the step; with prefill:false routing the trailing MAX_STEPS marker to role:user, thinking remains active and the model finishes its reasoning before yielding control.

Happy to close as duplicate of #26320 if maintainers prefer to consolidate the empty-filter discussion there and keep PR2 / PR3 (prefill schema + probe) under a fresh issue scoped to thinking-templates.

To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Navigation Menu

Repository navigation

Trailing-assistant 400 on llama.cpp/vLLM with thinking-on templates (Qwen3, DeepSeek-R1, GLM-thinking, Kimi-K2-Thinking, MiniMax-M2) #27920

Symptom

Reproduction

Affected model families

Upstream / cross-framework references

Why fix it in opencode

Proposed fix (three PRs)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Trailing-assistant 400 on llama.cpp/vLLM with thinking-on templates (Qwen3, DeepSeek-R1, GLM-thinking, Kimi-K2-Thinking, MiniMax-M2) #27920

Description

Symptom

Reproduction

Affected model families

Upstream / cross-framework references

Why fix it in opencode

Proposed fix (three PRs)

Activity

github-actions commented on May 16, 2026

feanor5555 commented on May 17, 2026

github-actions commented on Jul 17, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions