BUG: TemplateError

I keep running into these type of errors with most models I've tried (served by mlx-omni-server):

TemplateError: Conversation roles must alternate user/assistant/user/assistant/…
Thrown in transformers/utils/chat_template_utils.py:446.

mlx-omni-server` version: current HEAD, but this happening only in OpenCode not with other clients.

According to Kimi2, the issue is when a client sends two consecutive messages with the same role (e.g. two system messages), the server never reaches model memory allocation; it aborts during the Jinja2 chat-template pre-processing step with
RuntimeError: Stream generation failed: Conversation roles must alternate user/assistant/user/assistant/…

Partial log attached.

extract.log

This issue might be a duplicate of existing issues. Please check:

Duplicate System Message Sent #1301: Similar issue with duplicate system messages being sent, which can cause conversation role alternation errors with certain providers

Feel free to ignore if none of these address your specific case.

Ooops. Seems like it's the same issue. Though if there was a fix issued (in July?) it's still happening.

@oatmealm

Hello @oatmealm , could can you show how you configured this local model to run on opencode?

Hi there, right now I have this in opendoce.json (assuming that's what you need to see?):

opencode.json

Thanks, from what I can see your models or way you run them don't support multiple leading system messages, and I think the guys at Opencode would rather send 2 system messages as default to enable caching on systems like Anthropic. Still the AI SDK apparently has a way to check if a model allows this and if not it merges multiple system messages into 1 to prevent this error, but since you are running a local with a custom provider, it might think your models accept multiple system prompts when it doesn't.

This is what I can see for now, I am still doing some looking, best case right now would be to find the proper provider for this your way of running the models. Not sure why the @ai-sdk/openai-compatible one isn't merging the system prompts.

A long shot, but could you try this in your opencode config and let me know how it goes?

    "mlx-server": {
      "npm": "@ai-sdk/anthropic",
      "name": "MLX Omni Server (LAN)",
      "options": {
        "baseURL": "http://mbp.lan:10240/anthropic",
        "headers": {
          "Content-Type": "application/json"
        }
      },

Thanks. I was going to test that too :) Seems to work, though lots of other problems, like performance and irrelevant replys

@OpeOginni

did @OpeOginni help you get this solved or do you need further assistance?

@oatmealm

@oatmealm what I think is the best solution would be an AI SDK custom provider specifically for this way of running models. Maybe you can push the community to make one. But idk if it's the only solution anyone else with better clue can chip in.

Ultimately this seems like a provider / configuration issue rather than opencode, I can use kimi k2 through a variety of providers just fine

I will continue investigating with more models. I prefer local, though I'm wondering how realistic it is given context size requirements alone...?

if you want to use local models I recommend LLM Studio they seem to do best job in that department I always have people filing issues about ollama and others

Maybe I should. They do have full support for mlx now afaik but I thought the overhead of the gui or the single model standalone server (afaik it can't switch and load models after it's started) was too restrictive (i.e. I would need something else serving embeddings at least on the same machine.)

when you say not other clients what clients is it working in?

I was using emacs tools such as efrit (anthropic api as well) and gptel. But testing more trivial workflows with openwebui, I'm having too many problems with both mlx-omni-server and mlx-gui, i.e. embedding sometimes fails, models return 500 and even reboots (possibly realted to resource utiliaztion).

So I'm back on my original ollama setup, but of course the most immediate issue to tool calling support. I have some agents defined that don't need tools at all. I tried this:

        "deepseek-r1:8b": {
          "name": "DeepSeek R1 8B",
          "tools": false,
          "reasoning": true,
          "temperature": true,
          "options": {
            "temperature": 0.3
          }
        },

for this particular model, which afaik does not support tools at all, but opencode still complains:

AI_APICallError: registry.ollama.ai/library/deepseek-
r1:8b does not support tools

Same with

AI_APICallError: registry.ollama.ai/library/phi4-mini-
reasoning:latest does not support tools

Sorry, I see it's "tool_call": false, which works fine.
Question is if there's a way to use option to enable tools on Ollama models which support tool call by might need additional or specific options passed in?

Can you elaborate? I don't think I understand what you are asking there

[automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!

Navigation Menu

Repository navigation

BUG: TemplateError #2272

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

BUG: TemplateError #2272

Description

Activity

github-actions commented on Aug 27, 2025

oatmealm commented on Aug 27, 2025

OpeOginni commented on Aug 27, 2025

oatmealm commented on Aug 27, 2025

OpeOginni commented on Aug 27, 2025

OpeOginni commented on Aug 27, 2025

oatmealm commented on Aug 27, 2025

rekram1-node commented on Aug 27, 2025

OpeOginni commented on Aug 27, 2025

rekram1-node commented on Aug 27, 2025

oatmealm commented on Aug 27, 2025

rekram1-node commented on Aug 27, 2025

oatmealm commented on Aug 27, 2025

thdxr commented on Aug 27, 2025

oatmealm commented on Aug 29, 2025

oatmealm commented on Aug 29, 2025

rekram1-node commented on Aug 29, 2025

rekram1-node commented on Dec 28, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions