Multiple system prompts break Qwen3.5-* models

Description

As part of investigating what could've caused Tarquinen/opencode-dynamic-context-pruning#407 I noticed that the tool was adding another system prompt, which caused for this model to break.

Despite being fixed in the plugin it makes sense to at least have it as a known issue, so this is mostly for documentation purposes and if there are other plugins that could incur such behavior for the qwen3.5 or other models.

Also maybe as a suggestion if there are multiple system prompts to be concatenated into a single one before dispatch to the model, similarly to how it is done here

Plugins

@tarquinen/opencode-dcp

OpenCode version

1.2.12

Steps to reproduce

Install @tarquinen/opencode-dcp with version <= 2.1.7
Start a session with any of the Qwen3.5 models
Send a message and expect to receive error this error

Screenshot and/or share link

No response

Operating System

EndeavourOS

Terminal

Ghostty

I am noticing the same issue from my side as well. Here is my config:

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": [
    "opencode-websearch-cited@latest",
    "opencode-vibeguard@latest",
    "@franlol/opencode-md-table-formatter@latest"
  ],
  "provider": {
    "litellm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "LiteLLM",
      "options": {
        "baseURL": "https://litellm.example.com/v1"
      },
      "models": {
        "gondor/qwen3-coder-next:80b": {
          "name": "Gondor Qwen3 Coder Next"
        },
        "llama-swap/qwen3.5:122b-q4": {
          "name": "Gondor Qwen3.5 Q4 Generic"
        }
      }
    },
    "google": {
      "options": {
        "websearch_cited": {
          "model": "gemini-2.5-flash"
        }
      },
      "whitelist": [
        "gemini-2.5-flash",
        "gemini-2.5-pro",
        "gemini-3-flash-preview",
        "gemini-3-pro-preview",
        "gemini-3.1-pro-preview"
      ]
    }
  }
}

Here is the an extract of what litellm sees:

{
  "issue_type": "litellm_internal_server_error",
  "litellm_version": "1.81.14",
  "model_group": "gondor/qwen3.5:122b-q4",
  "deployment": "openai/qwen3.5:122b-q4",
  "status": "failure",
  "error": {
    "class": "InternalServerError",
    "code": "500",
    "message": "Jinja Exception: System message must be at the beginning."
  },
  "request_shape": {
    "endpoint_path": "/v1/chat/completions",
    "stream": true,
    "tool_choice": "auto",
    "tools_count": 14,
    "messages_order": [
      {
        "role": "system",
        "content_preview": "system prompt 1..."
      },
      {
        "role": "system",
        "content_preview": "system prompt 2..."
      },
      {
        "role": "user",
        "content_preview": "test"
      }
    ]
  },
  "request_excerpt": {
    "model": "gondor/qwen3.5:122b-q4",
    "top_p": 1,
    "stream": true,
    "max_tokens": 32000,
    "tool_choice": "auto",
    "tools_count": 14,
    "messages": [
      {
        "role": "system",
        "content_preview": "You are opencode, an interactive CLI tool..."
      },
      {
        "role": "system",
        "content_preview": "<EXTREMELY_IMPORTANT> You have superpowers..."
      },
      {
        "role": "user",
        "content_preview": "test"
      }
    ]
  },
  "retry_context": {
    "max_retries": 2,
    "attempted_retries": 2
  },
  "timing": {
    "latency_seconds": 1.7045
  },
  "sanitized_environment": {
    "api_base": "***REDACTED_INTERNAL_URL***",
    "endpoint": "***REDACTED_INTERNAL_URL***",
    "hostnames": "***REDACTED***",
    "ip_addresses": "***REDACTED***",
    "api_keys_tokens": "***REDACTED***",
    "user_identifiers": "***REDACTED***"
  }
}

I guess in my case superpowers is causing the issue?

yep, seems like it. I am not 100% sure what will be the correct approach, but I think that the plugin/mcp/etc adding those should be updated

I put together a candidate fix in my fork:

cyrilmhansen#2

This patch normalizes the final system prompt into a single system message before provider request construction.

It is aimed at cases like the one reported here with Qwen, where multiple system prompt fragments can accumulate from provider/user/plugin input and strict chat templates appear to be sensitive to that final message shape.

The problem is https://github.com/anomalyco/opencode/blob/dev/packages/opencode/src/session/llm.ts sends multiple system prompts and Qwen models(perhaps others) only supports one system prompt.
There's multiple open issues and PRs to fix this but don't understand what is the holdup.

@zhangdw156

✅ Confirmed: The system prompt merge fix works perfectly (#23656

I just tested this PR locally and can confirm it completely resolves the Jinja Exception: System message must be at the beginning error.

My setup:

Model: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive (Q4_K_M)
Server: llama-server with --jinja (required by this model's official template)
Client: OpenCode (built from this PR branch)
Context: 131072, tool calling enabled

With this patch, OpenCode correctly merges all system/developer messages into a single prompt before sending it to llama.cpp. The strict Jinja validation passes, and the model responds without breaking or hallucinating format.

🙏 Huge thanks to @zhangdw156 and the maintainers! This unblocks local Qwen3.x users who rely on strict chat templates.

❓ Quick question: Do you have an ETA for merging this into dev/main? Will it land in the next stable release (v1.14.x or v1.15.x)? I'm tracking it to update my global install as soon as it's available.

Happy to test edge cases (long context, multi-turn tool calling, vision+text, etc.) if needed. Keep up the amazing work! 🚀

I can reproduce this behaviour with Qwen3.5 architecture models like Qwen3.6-27B-GGUF in OpenCode v1.14.40.

The error message from the llama-server binary is:

Error: Jinja Exception: System message must be at the beginning.

But this is a strict limitation from the official template (https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/7da1103448ba36029c34ce1a9a741dfe93ee0c50/chat_template.jinja#L85) that can be fixed!

Workaround

Clone https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates and use llama-server with the following options:

--chat-template-file 'Qwen-Fixed-Chat-Templates\qwen3.6\chat_template.jinja'

--chat-template-kwargs '{"preserve_thinking":true}'

So maybe not a defect in OpenCode?

This works for me: https://github.com/countzero/windows_llama.cpp/blob/v1.30.0/presets/models_24GB_VRAM.ini#L179-L207

I can reproduce this behaviour with Qwen3.5 architecture models like Qwen3.6-27B-GGUF in OpenCode v1.14.40.

The error message from the llama-server binary is:

Error: Jinja Exception: System message must be at the beginning.

But this is a strict limitation from the official template (https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/7da1103448ba36029c34ce1a9a741dfe93ee0c50/chat_template.jinja#L85) that can be fixed!

Workaround

Clone https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates and use llama-server with the following options:

--chat-template-file 'Qwen-Fixed-Chat-Templates\qwen3.6\chat_template.jinja'

--chat-template-kwargs '{"preserve_thinking":true}'

So maybe not a defect in OpenCode?

This works for me: https://github.com/countzero/windows_llama.cpp/blob/v1.30.0/presets/models_24GB_VRAM.ini#L179-L207

Did this solution actually work for you? I remember trying to use that same chat template and adding it to my llamaCPP using the --chat-template-file argument, but it didn't work for me; my OpenCode kept giving me the same error.

@Milor123

@Milor123 yes it works:

You can check in the logs if the line srv params_from_: Chat format: peg-native is present. IMHO that indicates that a Jinja template is actually enabled and used.

complete llama-server log output for the above example request

ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24462 MiB):
Device 0: NVIDIA RTX PRO 5000 Blackwell Generation Laptop GPU, compute capability 12.0, VMM: yes, VRAM: 24462 MiB
main: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
build_info: b9053-97f06e9ee
system_info: n_threads = 24 (n_threads_batch = 24) / 24 | CUDA : ARCHS = 1200 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
init: using 23 threads for HTTP server
srv load_models: Loaded 0 cached model presets
srv load_models: Loaded 19 local model presets from D:\AI\LLM\gguf
srv load_models: Loaded 8 custom model presets from D:\Arbeit\windows_llama.cpp\presets\models_24GB_VRAM.ini
srv operator (): Available models (27) (*: custom preset)
srv operator (): * Abiray-Qwen3.6-27B-NVFP4+Qwen3.5-0.8B.IQ4_XS.gguf (aliases: Abiray-Qwen3.6-27B-NVFP4+Qwen3.5-0.8B.IQ4_XS.gguf)
srv operator (): Abiray-Qwen3.6-27B-NVFP4-GGUF
srv operator (): * Abiray-Qwen3.6-27B-NVFP4.gguf (aliases: Abiray-Qwen3.6-27B-NVFP4.gguf)
srv operator (): Bonsai-8B-gguf
srv operator (): Devstral-Small-2-24B-Instruct-2512
srv operator (): GLM-4.7-Flash
srv operator (): LFM2-24B-A2B
srv operator (): Qwen3-Coder-Next
srv operator (): * Qwen3-Coder-Next.IQ4_XS.gguf (aliases: Qwen3-Coder-Next.IQ4_XS.gguf)
srv operator (): Qwen3.5-0.8B
srv operator (): Qwen3.5-122B-A10B
srv operator (): Qwen3.5-27B
srv operator (): Qwen3.5-2B
srv operator (): Qwen3.5-35B-A3B
srv operator (): Qwen3.5-4B
srv operator (): Qwen3.5-9B
srv operator (): Qwen3.6-27B
srv operator (): * Qwen3.6-27B.IQ4_XS+Qwen3.5-0.8B.IQ4_XS.gguf (aliases: Qwen3.6-27B.IQ4_XS+Qwen3.5-0.8B.IQ4_XS.gguf)
srv operator (): * Qwen3.6-27B.IQ4_XS.gguf (aliases: Qwen3.6-27B.IQ4_XS.gguf)
srv operator (): Qwen3.6-35B-A3B
srv operator (): * Qwen3.6-35B-A3B.IQ4_XS.gguf (aliases: Qwen3.6-35B-A3B.IQ4_XS.gguf)
srv operator (): * Qwen3.6-35B-A3B.IQ4_XS.gguf+Qwen3.5-0.8B.IQ4_XS.gguf (aliases: Qwen3.6-35B-A3B.IQ4_XS+Qwen3.5-0.8B.IQ4_XS.gguf)
srv operator (): gemma-4-26B-A4B-it
srv operator (): gemma-4-31B-it
srv operator (): * gemma-4-31B-it.IQ4_XS.gguf (aliases: gemma-4-31B-it.IQ4_XS.gguf)
srv operator (): gpt-oss-120b-GGUF
srv operator (): medgemma-27b-it
main: starting router server, no model will be loaded in this process
start: binding port with default address family
main: router server is listening on http://0.0.0.0:8080
main: NOTE: router mode is experimental
main: it is not recommended to use this mode in untrusted environments
srv ensure_model: model name=Qwen3.6-35B-A3B.IQ4_XS.gguf is not loaded, loading...
srv load: spawning server instance with name=Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
srv load: spawning server instance with args:
srv load: D:\Arbeit\windows_llama.cpp\vendor\llama.cpp\build\bin\Release\llama-server.exe
srv load: --chat-template-file
srv load: vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template.jinja
srv load: --chat-template-kwargs
srv load: {"preserve_thinking":true}
srv load: --host
srv load: 127.0.0.1
srv load: --jinja
srv load: --min-p
srv load: 0.0
srv load: --mmproj-offload
srv load: --port
srv load: 54531
srv load: --presence-penalty
srv load: 1.5
srv load: --spec-ngram-mod-n-match
srv load: 24
srv load: --spec-ngram-mod-n-max
srv load: 64
srv load: --spec-ngram-mod-n-min
srv load: 48
srv load: --spec-type
srv load: ngram-mod
srv load: --temperature
srv load: 0.6
srv load: --top-k
srv load: 20
srv load: --top-p
srv load: 0.95
srv load: --alias
srv load: Qwen3.6-35B-A3B.IQ4_XS.gguf
srv load: --ctx-size
srv load: 262144
srv load: --cache-ram
srv load: 51200
srv load: --cache-type-k
srv load: q8_0
srv load: --cache-type-v
srv load: q8_0
srv load: --swa-checkpoints
srv load: 32
srv load: --flash-attn
srv load: true
srv load: --fit-target
srv load: 512
srv load: --kv-unified
srv load: --model
srv load: D:\AI\LLM\gguf\Qwen3.6-35B-A3B\Qwen3.6-35B-A3B.IQ4_XS.gguf
srv load: --mmproj
srv load: D:\AI\LLM\gguf\Qwen3.6-35B-A3B\mmproj.Qwen3.6-35B-A3B.BF16.gguf
srv load: --n-gpu-layers
srv load: -1
srv load: --parallel
srv load: 1
srv load: --reasoning
srv load: on
srv load: --threads
srv load: 24
srv ensure_model: waiting until model name=Qwen3.6-35B-A3B.IQ4_XS.gguf is fully loaded...
srv ensure_model: waiting until model name=Qwen3.6-35B-A3B.IQ4_XS.gguf is fully loaded...
[54531] ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24462 MiB):
[54531] Device 0: NVIDIA RTX PRO 5000 Blackwell Generation Laptop GPU, compute capability 12.0, VMM: yes, VRAM: 24462 MiB
[54531] build_info: b9053-97f06e9ee
[54531] system_info: n_threads = 24 (n_threads_batch = 24) / 24 | CUDA : ARCHS = 1200 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
[54531] init: using 23 threads for HTTP server
[54531] start: binding port with default address family
[54531] main: loading model
[54531] srv load_model: loading model 'D:\AI\LLM\gguf\Qwen3.6-35B-A3B\Qwen3.6-35B-A3B.IQ4_XS.gguf'
[54531] common_init_result: fitting params to device memory, for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on
[54531] common_params_fit_impl: getting device memory data for initial parameters:
[54531] common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
[54531] common_memory_breakdown_print: | - CUDA0 (RTX PRO 5000 Blackwell Generation Laptop GPU) | 24462 = 23087 + (21188 = 17593 + 2782 + 812) + -19812 |
[54531] common_memory_breakdown_print: | - Host | 777 = 257 + 0 + 520 |
[54531] common_params_fit_impl: projected to use 21188 MiB of device memory vs. 23087 MiB of free device memory
[54531] common_params_fit_impl: will leave 1898 >= 512 MiB of free device memory, no changes needed
[54531] common_fit_params: successfully fit params to free device memory
[54531] common_fit_params: fitting params to free memory took 0.87 seconds
[54531] llama_model_loader: loaded meta data with 48 key-value pairs and 733 tensors from D:\AI\LLM\gguf\Qwen3.6-35B-A3B\Qwen3.6-35B-A3B.IQ4_XS.gguf (version GGUF V3 (latest))
[54531] llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
[54531] llama_model_loader: - kv 0: general.architecture str = qwen35moe
[54531] llama_model_loader: - kv 1: general.type str = model
[54531] llama_model_loader: - kv 2: general.sampling.top_k i32 = 20
[54531] llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
[54531] llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
[54531] llama_model_loader: - kv 5: general.name str = Qwen3.6 35B A3B
[54531] llama_model_loader: - kv 6: general.basename str = Qwen3.6
[54531] llama_model_loader: - kv 7: general.size_label str = 35B-A3B
[54531] llama_model_loader: - kv 8: general.license str = apache-2.0
[54531] llama_model_loader: - kv 9: general.license.link str = https://huggingface.co/Qwen/Qwen3.6-3...
[54531] llama_model_loader: - kv 10: general.tags arr[str,1] = ["image-text-to-text"]
[54531] llama_model_loader: - kv 11: qwen35moe.block_count u32 = 40
[54531] llama_model_loader: - kv 12: qwen35moe.context_length u32 = 262144
[54531] llama_model_loader: - kv 13: qwen35moe.embedding_length u32 = 2048
[54531] llama_model_loader: - kv 14: qwen35moe.attention.head_count u32 = 16
[54531] llama_model_loader: - kv 15: qwen35moe.attention.head_count_kv u32 = 2
[54531] llama_model_loader: - kv 16: qwen35moe.rope.dimension_sections arr[i32,4] = [11, 11, 10, 0]
[54531] llama_model_loader: - kv 17: qwen35moe.rope.freq_base f32 = 10000000.000000
[54531] llama_model_loader: - kv 18: qwen35moe.attention.layer_norm_rms_epsilon f32 = 0.000001
[54531] llama_model_loader: - kv 19: qwen35moe.expert_count u32 = 256
[54531] llama_model_loader: - kv 20: qwen35moe.expert_used_count u32 = 8
[54531] llama_model_loader: - kv 21: qwen35moe.attention.key_length u32 = 256
[54531] llama_model_loader: - kv 22: qwen35moe.attention.value_length u32 = 256
[54531] llama_model_loader: - kv 23: qwen35moe.expert_feed_forward_length u32 = 512
[54531] llama_model_loader: - kv 24: qwen35moe.expert_shared_feed_forward_length u32 = 512
[54531] llama_model_loader: - kv 25: qwen35moe.ssm.conv_kernel u32 = 4
[54531] llama_model_loader: - kv 26: qwen35moe.ssm.state_size u32 = 128
[54531] llama_model_loader: - kv 27: qwen35moe.ssm.group_count u32 = 16
[54531] llama_model_loader: - kv 28: qwen35moe.ssm.time_step_rank u32 = 32
[54531] llama_model_loader: - kv 29: qwen35moe.ssm.inner_size u32 = 4096
[54531] llama_model_loader: - kv 30: qwen35moe.full_attention_interval u32 = 4
[54531] llama_model_loader: - kv 31: qwen35moe.rope.dimension_count u32 = 64
[54531] llama_model_loader: - kv 32: tokenizer.ggml.model str = gpt2
[54531] llama_model_loader: - kv 33: tokenizer.ggml.pre str = qwen35
[54531] llama_model_loader: - kv 34: tokenizer.ggml.tokens arr[str,248320] = ["!", """, "#", "$", "%", "&", "'", ...
[54531] llama_model_loader: - kv 35: tokenizer.ggml.token_type arr[i32,248320] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
[54531] llama_model_loader: - kv 36: tokenizer.ggml.merges arr[str,247587] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
[54531] llama_model_loader: - kv 37: tokenizer.ggml.eos_token_id u32 = 248046
[54531] llama_model_loader: - kv 38: tokenizer.ggml.padding_token_id u32 = 248044
[54531] llama_model_loader: - kv 39: tokenizer.ggml.bos_token_id u32 = 248044
[54531] llama_model_loader: - kv 40: tokenizer.ggml.add_bos_token bool = false
[54531] llama_model_loader: - kv 41: tokenizer.chat_template str = {%- set image_count = namespace(value...
[54531] llama_model_loader: - kv 42: general.quantization_version u32 = 2
[54531] llama_model_loader: - kv 43: general.file_type u32 = 30
[54531] llama_model_loader: - kv 44: quantize.imatrix.file str = D:\AI\LLM\imatrix\Qwen3.6-35B-A3B.imp...
[54531] llama_model_loader: - kv 45: quantize.imatrix.dataset str = D:\Arbeit\windows_llama.cpp\vendor\wi...
[54531] llama_model_loader: - kv 46: quantize.imatrix.entries_count u32 = 510
[54531] llama_model_loader: - kv 47: quantize.imatrix.chunks_count u32 = 20
[54531] llama_model_loader: - type f32: 301 tensors
[54531] llama_model_loader: - type q5_K: 40 tensors
[54531] llama_model_loader: - type q6_K: 1 tensors
[54531] llama_model_loader: - type iq4_xs: 391 tensors
[54531] print_info: file format = GGUF V3 (latest)
[54531] print_info: file type = IQ4_XS - 4.25 bpw
[54531] print_info: file size = 17.43 GiB (4.32 BPW)
[54531] llama_prepare_model_devices: using device CUDA0 (NVIDIA RTX PRO 5000 Blackwell Generation Laptop GPU) (0000:01:00.0) - 23151 MiB free
[54531] load: 0 unused tokens
[54531] load: printing all EOG tokens:
[54531] load: - 248044 ('<|endoftext|>')
[54531] load: - 248046 ('<|im_end|>')
[54531] load: - 248063 ('<|fim_pad|>')
[54531] load: - 248064 ('<|repo_name|>')
[54531] load: - 248065 ('<|file_sep|>')
[54531] load: special tokens cache size = 33
[54531] load: token to piece cache size = 1.7581 MB
[54531] print_info: arch = qwen35moe
[54531] print_info: vocab_only = 0
[54531] print_info: no_alloc = 0
[54531] print_info: n_ctx_train = 262144
[54531] print_info: n_embd = 2048
[54531] print_info: n_embd_inp = 2048
[54531] print_info: n_layer = 40
[54531] print_info: n_head = 16
[54531] print_info: n_head_kv = 2
[54531] print_info: n_rot = 64
[54531] print_info: n_swa = 0
[54531] print_info: is_swa_any = 0
[54531] print_info: n_embd_head_k = 256
[54531] print_info: n_embd_head_v = 256
[54531] print_info: n_gqa = 8
[54531] print_info: n_embd_k_gqa = 512
[54531] print_info: n_embd_v_gqa = 512
[54531] print_info: f_norm_eps = 0.0e+00
[54531] print_info: f_norm_rms_eps = 1.0e-06
[54531] print_info: f_clamp_kqv = 0.0e+00
[54531] print_info: f_max_alibi_bias = 0.0e+00
[54531] print_info: f_logit_scale = 0.0e+00
[54531] print_info: f_attn_scale = 0.0e+00
[54531] print_info: n_ff = 0
[54531] print_info: n_expert = 256
[54531] print_info: n_expert_used = 8
[54531] print_info: n_expert_groups = 0
[54531] print_info: n_group_used = 0
[54531] print_info: causal attn = 1
[54531] print_info: pooling type = -1
[54531] print_info: rope type = 40
[54531] print_info: rope scaling = linear
[54531] print_info: freq_base_train = 10000000.0
[54531] print_info: freq_scale_train = 1
[54531] print_info: n_ctx_orig_yarn = 262144
[54531] print_info: rope_yarn_log_mul = 0.0000
[54531] print_info: rope_finetuned = unknown
[54531] print_info: mrope sections = [11, 11, 10, 0]
[54531] print_info: ssm_d_conv = 4
[54531] print_info: ssm_d_inner = 4096
[54531] print_info: ssm_d_state = 128
[54531] print_info: ssm_dt_rank = 32
[54531] print_info: ssm_n_group = 16
[54531] print_info: ssm_dt_b_c_rms = 0
[54531] print_info: model type = 35B.A3B
[54531] print_info: model params = 34.66 B
[54531] print_info: general.name = Qwen3.6 35B A3B
[54531] print_info: vocab type = BPE
[54531] print_info: n_vocab = 248320
[54531] print_info: n_merges = 247587
[54531] print_info: BOS token = 248044 '<|endoftext|>'
[54531] print_info: EOS token = 248046 '<|im_end|>'
[54531] print_info: EOT token = 248046 '<|im_end|>'
[54531] print_info: PAD token = 248044 '<|endoftext|>'
[54531] print_info: LF token = 198 'Ċ'
[54531] print_info: FIM PRE token = 248060 '<|fim_prefix|>'
[54531] print_info: FIM SUF token = 248062 '<|fim_suffix|>'
[54531] print_info: FIM MID token = 248061 '<|fim_middle|>'
[54531] print_info: FIM PAD token = 248063 '<|fim_pad|>'
[54531] print_info: FIM REP token = 248064 '<|repo_name|>'
[54531] print_info: FIM SEP token = 248065 '<|file_sep|>'
[54531] print_info: EOG token = 248044 '<|endoftext|>'
[54531] print_info: EOG token = 248046 '<|im_end|>'
[54531] print_info: EOG token = 248063 '<|fim_pad|>'
[54531] print_info: EOG token = 248064 '<|repo_name|>'
[54531] print_info: EOG token = 248065 '<|file_sep|>'
[54531] print_info: max token length = 256
[54531] load_tensors: loading model tensors, this can take a while... (mmap = true, direct_io = false)
[54531] load_tensors: offloading output layer to GPU
[54531] load_tensors: offloading 39 repeating layers to GPU
[54531] load_tensors: offloaded 41/41 layers to GPU
[54531] load_tensors: CPU_Mapped model buffer size = 257.66 MiB
[54531] load_tensors: CUDA0 model buffer size = 17593.02 MiB
[54531] ...................................................................................................
[54531] common_init_result: added <|endoftext|> logit bias = -inf
[54531] common_init_result: added <|im_end|> logit bias = -inf
[54531] common_init_result: added <|fim_pad|> logit bias = -inf
[54531] common_init_result: added <|repo_name|> logit bias = -inf
[54531] common_init_result: added <|file_sep|> logit bias = -inf
[54531] llama_context: constructing llama_context
[54531] llama_context: n_seq_max = 1
[54531] llama_context: n_ctx = 262144
[54531] llama_context: n_ctx_seq = 262144
[54531] llama_context: n_batch = 2048
[54531] llama_context: n_ubatch = 512
[54531] llama_context: causal_attn = 1
[54531] llama_context: flash_attn = enabled
[54531] llama_context: kv_unified = true
[54531] llama_context: freq_base = 10000000.0
[54531] llama_context: freq_scale = 1
[54531] llama_context: CUDA_Host output buffer size = 0.95 MiB
[54531] llama_kv_cache: CUDA0 KV buffer size = 2720.00 MiB
[54531] llama_kv_cache: size = 2720.00 MiB (262144 cells, 10 layers, 1/1 seqs), K (q8_0): 1360.00 MiB, V (q8_0): 1360.00 MiB
[54531] llama_kv_cache: attn_rot_k = 1, n_embd_head_k_all = 256
[54531] llama_kv_cache: attn_rot_v = 1, n_embd_head_k_all = 256
[54531] llama_memory_recurrent: CUDA0 RS buffer size = 62.81 MiB
[54531] llama_memory_recurrent: size = 62.81 MiB ( 1 cells, 40 layers, 1 seqs), R (f32): 2.81 MiB, S (f32): 60.00 MiB
[54531] sched_reserve: reserving ...
[54531] sched_reserve: resolving fused Gated Delta Net support:
[54531] sched_reserve: fused Gated Delta Net (autoregressive) enabled
[54531] sched_reserve: fused Gated Delta Net (chunked) enabled
[54531] sched_reserve: CUDA0 compute buffer size = 812.28 MiB
[54531] sched_reserve: CUDA_Host compute buffer size = 520.29 MiB
[54531] sched_reserve: graph nodes = 3849
[54531] sched_reserve: graph splits = 2
[54531] sched_reserve: reserve took 68.80 ms, sched copies = 1
[54531] common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
[54531] clip_model_loader: model name: Qwen3.6 35B A3B
[54531] clip_model_loader: description:
[54531] clip_model_loader: GGUF version: 3
[54531] clip_model_loader: alignment: 32
[54531] clip_model_loader: n_tensors: 334
[54531] clip_model_loader: n_kv: 29
[54531]
[54531] clip_model_loader: has vision encoder
[54531] clip_ctx: CLIP using CUDA0 backend
[54531] load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
[54531] load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
[54531] load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842
[54531]
[54531] load_hparams: projector: qwen3vl_merger
[54531] load_hparams: n_embd: 1152
[54531] load_hparams: n_head: 16
[54531] load_hparams: n_ff: 4304
[54531] load_hparams: n_layer: 27
[54531] load_hparams: ffn_op: gelu
[54531] load_hparams: projection_dim: 2048
[54531]
[54531] --- vision hparams ---
[54531] load_hparams: image_size: 768
[54531] load_hparams: patch_size: 16
[54531] load_hparams: has_llava_proj: 0
[54531] load_hparams: minicpmv_version: 0
[54531] load_hparams: n_merge: 2
[54531] load_hparams: n_wa_pattern: 0
[54531] load_hparams: image_min_pixels: 8192
[54531] load_hparams: image_max_pixels: 4194304
[54531]
[54531] load_hparams: model size: 860.98 MiB
[54531] load_hparams: metadata size: 0.12 MiB
[54531] warmup: warmup with image size = 1472 x 1472
[54531] alloc_compute_meta: CUDA0 compute buffer size = 248.10 MiB
[54531] alloc_compute_meta: CPU compute buffer size = 24.93 MiB
[54531] alloc_compute_meta: graph splits = 1, nodes = 823
[54531] warmup: flash attention is enabled
[54531] srv load_model: loaded multimodal model, 'D:\AI\LLM\gguf\Qwen3.6-35B-A3B\mmproj.Qwen3.6-35B-A3B.BF16.gguf'
[54531] srv load_model: initializing slots, n_slots = 1
[54531] common_context_can_seq_rm: the target context does not support partial sequence removal
[54531] srv load_model: speculative decoding will use checkpoints
[54531] common_speculative_init: initialized ngram_mod with n_match=24, size=4194304 (16.000 MB)
[54531] slot load_model: id 0 | task -1 | speculative decoding context initialized
[54531] slot load_model: id 0 | task -1 | new slot, n_ctx = 262144
[54531] srv load_model: prompt cache is enabled, size limit: 51200 MiB
[54531] srv load_model: use --cache-ram 0 to disable the prompt cache
[54531] srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
[54531] srv init: init: idle slots will be saved to prompt cache and cleared upon starting a new task
[54531] init: chat template, example_format: '<|im_start|>system
[54531] You are a helpful assistant<|im_end|>
[54531] <|im_start|>user
[54531] Hello<|im_end|>
[54531] <|im_start|>assistant
[54531] Hi there<|im_end|>
[54531] <|im_start|>user
[54531] How are you?<|im_end|>
[54531] <|im_start|>assistant
[54531]
[54531] '
[54531] srv init: init: chat template, thinking = 1
[54531] main: model loaded
[54531] main: server is listening on http://127.0.0.1:54531
[54531] main: starting the main loop...
[54531] cmd_child_to_router:ready
srv proxy_reques: proxying request to model Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
srv proxy_reques: proxying request to model Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
[54531] srv operator (): child server monitoring thread started, waiting for EOF on stdin...
[54531] srv update_slots: all slots are idle
[54531] srv params_from_: Chat format: peg-native
[54531] slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
[54531] srv get_availabl: updating prompt cache
[54531] srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
[54531] srv update: - cache state: 0 prompts, 0.000 MiB (limits: 51200.000 MiB, 262144 tokens, 53687091200 est)
[54531] srv get_availabl: prompt cache update took 0.01 ms
[54531] slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> temp-ext -> dist
[54531] slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
[54531] slot update_slots: id 0 | task 0 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 865
[54531] slot update_slots: id 0 | task 0 | n_tokens = 0, memory_seq_rm [0, end)
[54531] slot update_slots: id 0 | task 0 | prompt processing progress, n_tokens = 349, batch.n_tokens = 349, progress = 0.403468
[54531] srv params_from_: Chat format: peg-native
[54531] slot update_slots: id 0 | task 0 | n_tokens = 349, memory_seq_rm [349, end)
[54531] slot update_slots: id 0 | task 0 | prompt processing progress, n_tokens = 861, batch.n_tokens = 512, progress = 0.995376
[54531] slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 348, pos_max = 348, n_tokens = 349, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 0 | n_tokens = 861, memory_seq_rm [861, end)
[54531] slot init_sampler: id 0 | task 0 | init sampler, took 0.09 ms, tokens: text = 865, total = 865
[54531] slot update_slots: id 0 | task 0 | prompt processing done, n_tokens = 865, batch.n_tokens = 4
[54531] slot create_check: id 0 | task 0 | created context checkpoint 2 of 32 (pos_min = 860, pos_max = 860, n_tokens = 861, size = 62.813 MiB)
[54531] begin: ngram_mod occupancy = 841/4194304 (0.00)
[54531] srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[54531] slot print_timing: id 0 | task 0 |
[54531] prompt eval time = 408.23 ms / 865 tokens ( 0.47 ms per token, 2118.92 tokens per second)
[54531] eval time = 2170.49 ms / 286 tokens ( 7.59 ms per token, 131.77 tokens per second)
[54531] total time = 2578.72 ms / 1151 tokens
[54531] statistics ngram_mod: #calls(b,g,a) = 1 285 0, #gen drafts = 0, #acc drafts = 0, #gen tokens = 0, #acc tokens = 0, dur(b,g,a) = 0.102, 0.267, 0.000 ms
[54531] slot release: id 0 | task 0 | stop processing: n_tokens = 1150, truncated = 0
[54531] slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 19491464
[54531] srv get_availabl: updating prompt cache
[54531] srv prompt_save: - saving prompt with length 1150, total state size = 74.768 MiB
[54531] srv load: - looking for better prompt, base f_keep = 0.003, sim = 0.000
[54531] srv update: - cache state: 1 prompts, 200.394 MiB (limits: 51200.000 MiB, 262144 tokens, 293820 est)
[54531] srv update: - prompt 000001C5B491D500: 1150 tokens, checkpoints: 2, 200.394 MiB
[54531] srv get_availabl: prompt cache update took 57.04 ms
[54531] reasoning-budget: activated, budget=2147483647 tokens
[54531] slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> temp-ext -> dist
[54531] slot launch_slot_: id 0 | task 2 | processing task, is_child = 0
[54531] slot update_slots: id 0 | task 2 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 30189
[54531] slot update_slots: id 0 | task 2 | n_past = 3, slot.prompt.tokens.size() = 1150, seq_id = 0, pos_min = 1149, n_swa = 0
[54531] slot update_slots: id 0 | task 2 | Checking checkpoint with [860, 860] against 3...
[54531] slot update_slots: id 0 | task 2 | Checking checkpoint with [348, 348] against 3...
[54531] slot update_slots: id 0 | task 2 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
[54531] slot update_slots: id 0 | task 2 | erased invalidated context checkpoint (pos_min = 348, pos_max = 348, n_tokens = 349, n_swa = 0, pos_next = 0, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | erased invalidated context checkpoint (pos_min = 860, pos_max = 860, n_tokens = 861, n_swa = 0, pos_next = 0, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 0, memory_seq_rm [0, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.067839
[54531] slot update_slots: id 0 | task 2 | n_tokens = 2048, memory_seq_rm [2048, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.135679
[54531] slot update_slots: id 0 | task 2 | n_tokens = 4096, memory_seq_rm [4096, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 6144, batch.n_tokens = 2048, progress = 0.203518
[54531] slot update_slots: id 0 | task 2 | n_tokens = 6144, memory_seq_rm [6144, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 2048, progress = 0.271357
[54531] slot update_slots: id 0 | task 2 | n_tokens = 8192, memory_seq_rm [8192, end)
[54531] slot update_slots: id 0 | task 2 | 8192 tokens since last checkpoint at 0, creating new checkpoint during processing at position 10240
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 10240, batch.n_tokens = 2048, progress = 0.339196
[54531] slot create_check: id 0 | task 2 | created context checkpoint 1 of 32 (pos_min = 8191, pos_max = 8191, n_tokens = 8192, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 10240, memory_seq_rm [10240, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 12288, batch.n_tokens = 2048, progress = 0.407036
[54531] slot update_slots: id 0 | task 2 | n_tokens = 12288, memory_seq_rm [12288, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 14336, batch.n_tokens = 2048, progress = 0.474875
[54531] slot update_slots: id 0 | task 2 | n_tokens = 14336, memory_seq_rm [14336, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 16384, batch.n_tokens = 2048, progress = 0.542714
[54531] slot update_slots: id 0 | task 2 | n_tokens = 16384, memory_seq_rm [16384, end)
[54531] slot update_slots: id 0 | task 2 | 8192 tokens since last checkpoint at 8192, creating new checkpoint during processing at position 18432
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 18432, batch.n_tokens = 2048, progress = 0.610554
[54531] slot create_check: id 0 | task 2 | created context checkpoint 2 of 32 (pos_min = 16383, pos_max = 16383, n_tokens = 16384, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 18432, memory_seq_rm [18432, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 20480, batch.n_tokens = 2048, progress = 0.678393
[54531] slot update_slots: id 0 | task 2 | n_tokens = 20480, memory_seq_rm [20480, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 22528, batch.n_tokens = 2048, progress = 0.746232
[54531] slot update_slots: id 0 | task 2 | n_tokens = 22528, memory_seq_rm [22528, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 24576, batch.n_tokens = 2048, progress = 0.814071
[54531] slot update_slots: id 0 | task 2 | n_tokens = 24576, memory_seq_rm [24576, end)
[54531] slot update_slots: id 0 | task 2 | 8192 tokens since last checkpoint at 16384, creating new checkpoint during processing at position 26624
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 26624, batch.n_tokens = 2048, progress = 0.881911
[54531] slot create_check: id 0 | task 2 | created context checkpoint 3 of 32 (pos_min = 24575, pos_max = 24575, n_tokens = 24576, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 26624, memory_seq_rm [26624, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 28672, batch.n_tokens = 2048, progress = 0.949750
[54531] slot update_slots: id 0 | task 2 | n_tokens = 28672, memory_seq_rm [28672, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 29673, batch.n_tokens = 1001, progress = 0.982908
[54531] slot update_slots: id 0 | task 2 | n_tokens = 29673, memory_seq_rm [29673, end)
[54531] slot update_slots: id 0 | task 2 | prompt processing progress, n_tokens = 30185, batch.n_tokens = 512, progress = 0.999867
[54531] slot create_check: id 0 | task 2 | created context checkpoint 4 of 32 (pos_min = 29672, pos_max = 29672, n_tokens = 29673, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 2 | n_tokens = 30185, memory_seq_rm [30185, end)
[54531] slot init_sampler: id 0 | task 2 | init sampler, took 2.85 ms, tokens: text = 30189, total = 30189
[54531] slot update_slots: id 0 | task 2 | prompt processing done, n_tokens = 30189, batch.n_tokens = 4
[54531] slot create_check: id 0 | task 2 | created context checkpoint 5 of 32 (pos_min = 30184, pos_max = 30184, n_tokens = 30185, size = 62.813 MiB)
[54531] begin: ngram_mod occupancy = 24353/4194304 (0.01)
[54531] ~llama_io_write_device: allocated 'CUDA0' buffer 62.812 MiB
[54531] srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[54531] reasoning-budget: deactivated (natural end)
[54531] slot print_timing: id 0 | task 2 |
[54531] prompt eval time = 11103.66 ms / 30189 tokens ( 0.37 ms per token, 2718.83 tokens per second)
[54531] eval time = 895.26 ms / 84 tokens ( 10.66 ms per token, 93.83 tokens per second)
[54531] total time = 11998.92 ms / 30273 tokens
[54531] draft acceptance rate = 0.03125 ( 2 accepted / 64 generated)
[54531] statistics ngram_mod: #calls(b,g,a) = 2 366 1, #gen drafts = 1, #acc drafts = 1, #gen tokens = 64, #acc tokens = 2, dur(b,g,a) = 2.067, 0.353, 0.000 ms
[54531] slot release: id 0 | task 2 | stop processing: n_tokens = 30272, truncated = 0
[54531] srv update_slots: all slots are idle
srv proxy_reques: proxying request to model Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
[54531] srv params_from_: Chat format: peg-native
[54531] slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.996 (> 0.100 thold), f_keep = 1.000
[54531] reasoning-budget: activated, budget=2147483647 tokens
[54531] slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> temp-ext -> dist
[54531] slot launch_slot_: id 0 | task 389 | processing task, is_child = 0
[54531] slot update_slots: id 0 | task 389 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 30404
[54531] slot update_slots: id 0 | task 389 | n_tokens = 30272, memory_seq_rm [30272, end)
[54531] slot update_slots: id 0 | task 389 | prompt processing progress, n_tokens = 30400, batch.n_tokens = 128, progress = 0.999868
[54531] slot create_check: id 0 | task 389 | created context checkpoint 6 of 32 (pos_min = 30271, pos_max = 30271, n_tokens = 30272, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 389 | n_tokens = 30400, memory_seq_rm [30400, end)
[54531] slot init_sampler: id 0 | task 389 | init sampler, took 2.92 ms, tokens: text = 30404, total = 30404
[54531] slot update_slots: id 0 | task 389 | prompt processing done, n_tokens = 30404, batch.n_tokens = 4
[54531] slot create_check: id 0 | task 389 | created context checkpoint 7 of 32 (pos_min = 30399, pos_max = 30399, n_tokens = 30400, size = 62.813 MiB)
[54531] begin: ngram_mod occupancy = 24565/4194304 (0.01)
[54531] srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[54531] reasoning-budget: deactivated (natural end)
[54531] slot print_timing: id 0 | task 389 |
[54531] prompt eval time = 162.66 ms / 132 tokens ( 1.23 ms per token, 811.53 tokens per second)
[54531] eval time = 575.59 ms / 47 tokens ( 12.25 ms per token, 81.66 tokens per second)
[54531] total time = 738.25 ms / 179 tokens
[54531] draft acceptance rate = 0.03125 ( 2 accepted / 64 generated)
[54531] statistics ngram_mod: #calls(b,g,a) = 3 410 2, #gen drafts = 2, #acc drafts = 2, #gen tokens = 128, #acc tokens = 4, dur(b,g,a) = 4.041, 0.413, 0.000 ms
[54531] slot release: id 0 | task 389 | stop processing: n_tokens = 30450, truncated = 0
[54531] srv update_slots: all slots are idle
srv proxy_reques: proxying request to model Qwen3.6-35B-A3B.IQ4_XS.gguf on port 54531
[54531] srv params_from_: Chat format: peg-native
[54531] slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.885 (> 0.100 thold), f_keep = 1.000
[54531] reasoning-budget: activated, budget=2147483647 tokens
[54531] slot launch_slot_: id 0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> ?top-p -> ?min-p -> ?xtc -> temp-ext -> dist
[54531] slot launch_slot_: id 0 | task 437 | processing task, is_child = 0
[54531] slot update_slots: id 0 | task 437 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 34412
[54531] slot update_slots: id 0 | task 437 | n_tokens = 30450, memory_seq_rm [30450, end)
[54531] slot update_slots: id 0 | task 437 | prompt processing progress, n_tokens = 32498, batch.n_tokens = 2048, progress = 0.944380
[54531] slot update_slots: id 0 | task 437 | n_tokens = 32498, memory_seq_rm [32498, end)
[54531] slot update_slots: id 0 | task 437 | prompt processing progress, n_tokens = 33896, batch.n_tokens = 1398, progress = 0.985005
[54531] slot update_slots: id 0 | task 437 | n_tokens = 33896, memory_seq_rm [33896, end)
[54531] slot update_slots: id 0 | task 437 | prompt processing progress, n_tokens = 34408, batch.n_tokens = 512, progress = 0.999884
[54531] slot create_check: id 0 | task 437 | created context checkpoint 8 of 32 (pos_min = 33895, pos_max = 33895, n_tokens = 33896, size = 62.813 MiB)
[54531] slot update_slots: id 0 | task 437 | n_tokens = 34408, memory_seq_rm [34408, end)
[54531] slot init_sampler: id 0 | task 437 | init sampler, took 3.25 ms, tokens: text = 34412, total = 34412
[54531] slot update_slots: id 0 | task 437 | prompt processing done, n_tokens = 34412, batch.n_tokens = 4
[54531] slot create_check: id 0 | task 437 | created context checkpoint 9 of 32 (pos_min = 34407, pos_max = 34407, n_tokens = 34408, size = 62.813 MiB)
[54531] begin: ngram_mod occupancy = 28493/4194304 (0.01)
[54531] srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
srv log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
[54531] reasoning-budget: deactivated (natural end)
[54531] slot print_timing: id 0 | task 437 |
[54531] prompt eval time = 1684.01 ms / 3962 tokens ( 0.43 ms per token, 2352.71 tokens per second)
[54531] eval time = 2114.12 ms / 209 tokens ( 10.12 ms per token, 98.86 tokens per second)
[54531] total time = 3798.13 ms / 4171 tokens
[54531] statistics ngram_mod: #calls(b,g,a) = 4 618 2, #gen drafts = 2, #acc drafts = 2, #gen tokens = 128, #acc tokens = 4, dur(b,g,a) = 6.291, 0.634, 0.000 ms
[54531] slot release: id 0 | task 437 | stop processing: n_tokens = 34620, truncated = 0
[54531] srv update_slots: all slots are idle

It dont work for me. I need use the opencode patch or not work in the stable release

log error

clip_model_loader: has vision encoder
clip_ctx: CLIP using CUDA0 backend
load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842

load_hparams: projector:          qwen3vl_merger
load_hparams: n_embd:             1152
load_hparams: n_head:             16
load_hparams: n_ff:               4304
load_hparams: n_layer:            27
load_hparams: ffn_op:             gelu
load_hparams: projection_dim:     2048

--- vision hparams ---
load_hparams: image_size:         768
load_hparams: patch_size:         16
load_hparams: has_llava_proj:     0
load_hparams: minicpmv_version:   0
load_hparams: n_merge:            2
load_hparams: n_wa_pattern: 0
load_hparams: image_min_pixels:   8192
load_hparams: image_max_pixels:   4194304

load_hparams: model size:         860.98 MiB
load_hparams: metadata size:      0.12 MiB
warmup: warmup with image size = 1472 x 1472
alloc_compute_meta:      CUDA0 compute buffer size =   248.10 MiB
alloc_compute_meta:        CPU compute buffer size =    24.93 MiB
alloc_compute_meta: graph splits = 1, nodes = 823
warmup: flash attention is enabled
srv    load_model: loaded multimodal model, 'C:\Users\User\.lmstudio\models\mudler\Qwen3.6-35B-A3B-uncensored-heretic-APEX-GGUF\mmproj.gguf'
srv    load_model: initializing slots, n_slots = 1
common_speculative_is_compat: the target context does not support partial sequence removal
srv    load_model: speculative decoding not supported by this context
slot   load_model: id  0 | task -1 | new slot, n_ctx = 262144
srv    load_model: prompt cache is enabled, size limit: 8192 MiB
srv    load_model: use `--cache-ram 0` to disable the prompt cache
srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
srv          init: init: idle slots will be saved to prompt cache and cleared upon starting a new task
init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
srv          init: init: chat template, thinking = 1
main: model loaded
main: server is listening on http://127.0.0.1:10000
main: starting the main loop...
srv  update_slots: all slots are idle
srv   operator (): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 116, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 500
srv   operator (): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 116, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 500
srv   operator (): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 116, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 500
                                                                                      srv   operator ():
got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 116, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 500
srv   operator (): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 116, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 500

input
.\llama-server.exe -m "C:\Users\User\.lmstudio\models\mudler\Qwen3.6-35B-A3B-uncensored-heretic-APEX-GGUF\Qwen3.6-35B-A3B-uncensored-heretic-APEX-I-Quality.gguf" --mmproj "C:\Users\User\.lmstudio\models\mudler\Qwen3.6-35B-A3B-uncensored-heretic-APEX-GGUF\mmproj.gguf" --host 127.0.0.1 --port 10000 -ngl 99 --n-cpu-moe 28 -c 262144 -t 8 -tb 14 -b 2048 -ub 1024 -np 1 -fa on -ctk turbo4 -ctv turbo3 --top-k 20 --repeat-penalty 1.0 --top-p 0.95 --min-p 0.0 -fit off --mlock --no-mmap -kvu --chat-template-kwargs '{"preserve_thinking":true}' --chat-template-file "S:\DownloadsWIN\chat_template.jinja" --jinja

@Milor123

@Milor123

The error is coming from your chat template. Your S:\DownloadsWIN\chat_template.jinja still has the upstream guard at line 116:

{%- if not loop.first %}
{{- raise_exception('System message must be at the beginning.') }}

Interestingly there has been another update to the fixed template. I would now suggest you use the following:

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/781cc5152b46bacba17fd1aadbea5049d8d490f1/qwen3.6/chat_template-v8.jinja

Will also test that one once I find some time 😉

@adamdotdevin

still reproducible on opencode v1.14.39 against Qwen/Qwen3.6-27B and Qwen/Qwen3.6-35B-A3B on vLLM v0.21.0.

also fires with no plugins. opencode's title-generator path emits two role: "system" entries on the first call of any session (title-generator prompt + inherited delegation block), captured from the LiteLLM proxy spend log:

[0] system: "You are a title generator. You output ONLY a thread title..." 

[1] system: "<task-notification><delegation-system> ## Async Delegation..." 

[2] user: "Generate a title for this conversation:" 

[3] user: "<actual user input>"

repro without opencode in the path:

curl -sS http://VLLM_HOST:8000/v1/chat/completions
-H 'Content-Type: application/json'
-d '{ "model": "Qwen/Qwen3.6-35B-A3B", "messages": [ {"role": "system", "content": "A"}, {"role": "system", "content": "B"}, {"role": "user", "content": "hi"} ], "max_tokens": 8 }'

{"error":{"message":"System message must be at the beginning.","type":"BadRequestError","code":400}}
template clause: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/raw/main/chat_template.jinja (search loop.first + raise_exception).

current build site is packages/opencode/src/session/llm/request.ts ~line 92:

const messages = isOpenaiOauth || input.isWorkflow ? input.messages : [ ...system.map((x): ModelMessage => ({ role: "system", content: x })), ...input.messages, ]

collapsing the array resolves it:

const messages = isOpenaiOauth || input.isWorkflow ? input.messages : [ ...(system.length > 0 ? [{ role: "system" as const, content: system.join("\n\n") }] : []), ...input.messages, ]

#16981 generalizes this with a systemMessage: 'single' | 'multiple' capability and is currently waiting on @adamdotdevin review.

Still reproducing on opencode 1.15.11 with a different plugin (opencode-goal-plugin@0.1.8), so dropping a fresh repro here in case it helps maintainers prioritize the coalescer fix.

Repro

opencode 1.15.11, plugin opencode-goal-plugin@0.1.8.
Provider: @ai-sdk/openai-compatible -> self-hosted vLLM serving Qwen3.5-122B-A10B with the upstream chat template.
Run /goal <anything> in a session.
Request fails immediately with AI_APICallError and response body "System message must be at the beginning.".

Wire payload (from opencode's own debug log)

"messages": [
  { "role": "system", "content": "You are opencode, an interactive CLI tool ..." },
  { "role": "system", "content": "<goal_continuation>\n...\n<goal_objective>\ndraw a ball\n</goal_objective>\n..." },
  { "role": "user",   "content": "draw a ball" }
]

Two role: "system" messages. The Qwen3.5 chat template raises on the second one (raise_exception('System message must be at the beginning.')), vLLM returns 400, opencode surfaces it as AI_APICallError.

Why this is opencode's bug (not just the plugin's)

The compiled binary shows opencode intends to coalesce plugin-added system blocks after experimental.chat.system.transform fires, but the guard is off-by-one — coalescing only runs when system.length > 2, so the common case of one plugin pushing one block (final length = 2) goes straight through to the provider as two separate role: "system" messages.

Decompiled snippet from the 1.15.11 binary:

const o = r[0];
yield* A.plugin.trigger("experimental.chat.system.transform", ..., { system: r });
if (r.length > 2 && r[0] === o) {        // ← should be `> 1`
    const C = r.slice(1);
    r.length = 0;
    r.push(o, C.join("\n"));
}

Until this is fixed upstream, any plugin doing output.system.push(...) (the documented hook usage pattern) silently breaks strict-template backends.

Candidate fix already exists

cyrilmhansen#2 was merged to that fork on 2026-03-26 with exactly the right shape — collapse system fragments into a single newline-joined system message before provider request construction. It does not appear to have landed on anomalyco/opencode dev yet, and 1.15.11 still ships the broken coalescer. Any blockers to rebasing and merging it?

Plugin-side report filed in parallel

I've also filed william-ricchiuti/OpenCode-goal-plugin#1 asking the plugin to merge into system[0] defensively instead of push-ing, per the guidance requested in #23660. That fix is independent and gives users on strict backends a path forward without waiting on the upstream coalescer.

Environment

opencode: 1.15.11
Plugin: opencode-goal-plugin@0.1.8
Provider: @ai-sdk/openai-compatible
Backend: vLLM 0.19.x, Qwen3.5-122B-A10B Hybrid INT4/FP8
OS: Linux 6.17 aarch64 (NVIDIA GB10)

Quick A/B validation of the plugin-side workaround against my reproducer (opencode 1.15.11 + goal-plugin 0.1.8 + Qwen3.5-122B vLLM):

Plugin code	Result
`output.system.push(...)` (current goal-plugin behavior)	HTTP 400 `"System message must be at the beginning."`
Merge into `output.system[0]` (patched locally in the plugin file)	Request succeeded, model executed tools, session continued normally

This confirms both:

The user-visible bug is avoidable by plugins today if they follow the documented merge pattern (per Clarify plugin guidance for experimental.chat.system.transform to avoid extra system messages #23660).
But it shouldn't have to be — every plugin author would have to learn this gotcha. The right durable fix is still the coalescer change in opencode core (fix(opencode): collapse system prompt fragments into a single system message cyrilmhansen/opencode#2 shape), so plugins can keep using the natural output.system.push pattern that the hook signature suggests.

Tracking the plugin-side fix at william-ricchiuti/OpenCode-goal-plugin#1 to unblock users now while the upstream coalescer fix lands.

I'm running into this as well, not sure how to debug.

I see this issue with some models, now with bottlecapai/ThinkingCap-Qwen3.6-27B-NVFP4 on vLLM.

on chutes/Qwen/Qwen3.6-27B-TEE, I'm getting:

Bad Request: {"detail":"Invalid request: Invalid request: {\"detail\":{\"object\":\"error\",\"message\":\"System message must be at the beginning.\",\"type\":\"BadRequest\",\"param\":null,\"code\":400}}"}

Attempting to use locally hosted model via Nativ.app, mlx-community/Qwen3.6-35B-A3B-8bit, I get on the server side:

2026-08-12 11:41:13,147 - ERROR - Unexpected error in /chat/completions endpoint: System message must be at the beginning.

Nativ.app more info: https://github.com/Blaizzy/nativ

Thanks for the report! OpenCode 2.0 is now out and development is focused on V2, so we're closing issues filed against OpenCode 1. Please upgrade (opencode upgrade --major, details: https://anoma.ly/notes/opencode-2-0), and if this is still relevant on 2.0, reopen this issue or open a new one with your 2.0 version.

— from 𝕺𝖕𝖊𝖓𝕮𝖔𝖉𝖊

Navigation Menu

Repository navigation

Multiple system prompts break Qwen3.5-* models #15059

Description

Plugins

OpenCode version

Steps to reproduce

Screenshot and/or share link

Operating System

Terminal

✅ Confirmed: The system prompt merge fix works perfectly (#23656

Repro

Wire payload (from opencode's own debug log)

Why this is opencode's bug (not just the plugin's)

Candidate fix already exists

Plugin-side report filed in parallel

Environment

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Multiple system prompts break Qwen3.5-* models #15059

Description

Description

Plugins

OpenCode version

Steps to reproduce

Screenshot and/or share link

Operating System

Terminal

Activity

marcobox commented on Mar 8, 2026

DaGhostman commented on Mar 9, 2026

cyrilmhansen commented on Mar 26, 2026

mingfang commented on Apr 16, 2026

Milor123 commented on Apr 26, 2026

✅ Confirmed: The system prompt merge fix works perfectly (#23656

countzero commented on May 7, 2026

Milor123 commented on May 7, 2026

countzero commented on May 7, 2026

Milor123 commented on May 7, 2026

countzero commented on May 8, 2026

robertoberto commented on May 23, 2026

fank commented on May 27, 2026

Repro

Wire payload (from opencode's own debug log)

Why this is opencode's bug (not just the plugin's)

Candidate fix already exists

Plugin-side report filed in parallel

Environment

OneOfOne commented on Jun 3, 2026

SurealCereal commented on Aug 6, 2026

zcutlip commented on Aug 12, 2026

thdxr commented on Oct 11, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions