Tool use with Ollama models.

using latest ollama version and tool capable models, executions of tool calls seems to be aborted.

Models are not keen to use tool calls, but could be encourage to at least attempt for one but result fails.

opencode.json file;

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen3:30b": {
            "name": "Qwen3:30b 65k MoE - local",
            "tools": true,
            "reasoning": true,
            "options": { "num_ctx": 65536 }
        },
        "devstral": {
            "name": "Devstral 14b Dense - local",
            "tools": true,
            "reasoning": false,
            "options": { "num_ctx": 131072 }
        }
      }
    },
    "moonshot": {
      "id": "moonshot",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "https://api.moonshot.cn/v1"
       },
      "models": {
        "kimi-k2-0711-preview": {
          "name": "kimi-k2-0711-preview",
          "cost": {
            "input": 0.001953125,
            "output": 0.009765625
          }
        }
      }
    }
  }
}

My hack to have this work on NixOS.

{ pkgs, ... }:

let
  opencodeBin = "/home/berkay/.opencode/bin/opencode";
  configSource = "/home/berkay/.config/opencode/providers/opencode.json";
  authSource = "/home/berkay/.local/share/opencode/auth.json";
  wrapper = pkgs.writeShellScriptBin "opencode" ''
    #!${pkgs.bash}/bin/bash

    CONFIG_SOURCE="${configSource}"
    CONFIG_LINK="./opencode.json"
    AUTH_SOURCE="${authSource}"
    AUTH_LINK="./auth.json"
    
    # Symlink config if not present
    CLEANUP_CONFIG_SYMLINK=0
    if [ ! -f "$CONFIG_LINK" ]; then
      ln -s "$CONFIG_SOURCE" "$CONFIG_LINK"
      CLEANUP_CONFIG_SYMLINK=1
    fi
    # Symlink auth if not present
    CLEANUP_AUTH_SYMLINK=0
    if [ ! -f "$AUTH_LINK" ]; then
      ln -s "$AUTH_SOURCE" "$AUTH_LINK"
      CLEANUP_AUTH_SYMLINK=1
    fi
    # Run opencode
    if [ -x "${opencodeBin}" ]; then
      ${pkgs.steam-run}/bin/steam-run "${opencodeBin}" "$@"
      STATUS=$?
    else
      echo "Error: opencode binary not found at ${opencodeBin}"
      exit 1
    fi
    # Remove symlinks if we created them
    if [ "$CLEANUP_CONFIG_SYMLINK" = "1" ]; then
      rm -f "$CONFIG_LINK"
    fi
    if [ "$CLEANUP_AUTH_SYMLINK" = "1" ]; then
      rm -f "$AUTH_LINK"
    fi
    exit $STATUS
  '';
in
{
  home.packages = [
    pkgs.steam-run
    wrapper
  ];

  xdg.desktopEntries.opencode = {
    name = "OpenCode";
    comment = "AI Coding Agent TUI";
    exec = "opencode";
    icon = "utilities-terminal";
    categories = [ "Development" ];
    terminal = true;
    startupNotify = false;
  };

  xdg.enable = true;
}

#1034 - same issue, frustrating.

Not a solution, but I have had better success using LMStudio. Important with that one is that you increase context scope of the model

"options": { "num_ctx": 131072 }
Does this work with ollama ? I think this is not supported with openai-compatible ollama/ollama#8938

"options": { "num_ctx": 131072 } Does this work with ollama ? I think this is not supported with openai-compatible ollama/ollama#8938

Where exactly does the num_ctx field go? I am not really sure where that should go.

@p-lemonish

OK. ollama user here in Fedora 42.

I was looking for this solution for a while. This is why I'm posting it here (and all issues) for all to know:

@p-lemonish discovered why and the fix is easy: https://github.com/p-lemonish/ollama-x-opencode

In short, download a model that has tool capabilities:

$ ollama show granite3.3:8b
  Model
    architecture        granite    
    parameters          8.2B       
    context length      131072     
    embedding length    4096       
    quantization        Q4_K_M     

  Capabilities
    completion    
    tools         

  License
    Apache License               
    Version 2.0, January 2004    
    ...                          

And edit it so that it has a fixed num_ctx:

```terminal
$ ollama run qwen3:latest
>>> /set parameter num_ctx 16384
Set parameter 'num_ctx' to '16384'
>>> /save qwen3:latest
Created new model 'qwen3:latest'
>>> /bye

After that, you're good to go. Just try it out:

opencode run "generate a todo.md file with the contents 'hello world, from me' in it" --model ollama/qwen3:latest

With a bit of luck, your file will be there.

Credits:

https://github.com/p-lemonish/ollama-x-opencode

Pre-requisites

Your model must be downloaded and functional in ollama.
You need to configure your model in opencode.json

My configuration looks like this:

{
    "$schema": "https://opencode.ai/config.json",
    "provider": {
        "ollama": {
            "npm": "@ai-sdk/openai-compatible",
            "options": {
                "baseURL": "http://localhost:11434/v1"
            },
            "models": {
                "granite3.3:8b": {
                    "tools": true
                },
                "mistral-small3.2:24b": {
                    "tools": true
                },
                "qwen3:latest": {
                    "tools": true,
                    "reasoning": true
                }
            }
        }
    }
}

Notes

granite3.3 still doesn't work.

It seems they added OLLAMA_CONTEXT_LENGTH as an env var for Ollama. I'm testing it as we speak, but it seems like it works!

@renich

The >>> /set parameter num_ctx 16384 from @renich was the key for me. Once I set that on qwen3 the tool usage started working.

I have a sort of workaround that semi-works for now. The issue is not only the context of ollama, but also a model template. Here how this could be addressed

Pull models

ollama pull qwen3:30b
ollama pull qwen3-coder:30b

View template of existing model:

ollama show qwen3:30b --template

Create initial version of Modelfile-qwen3-coder.dat file (must be manually modified later):

ollama show qwen3:30b --template > Modelfile-qwen3-coder.dat

Edit file with content of the file Modelfile-qwen3-coder.dat manually.
Modelfile-qwen3-coder.dat must start with:

FROM qwen3-coder:30b
PARAMETER num_ctx 32768
TEMPLATE """

Modelfile-qwen3-coder.dat must end it:

"""

Create a new ollama model

ollama create qwen3-coder-32k -f Modelfile-qwen3-coder.dat

Then use qwen3-coder-32k in opencode

P.S. It is not a final solution. But maybe it will save some people a bit of time in searching and trying out different ways of making opencode work with ollama models

How does websearch tool work in ollama?

@BhautikChudasama

@BhautikChudasama the tool should just work assuming ollama has enough context set and is presenting it as an option to the llm

How does websearch tool work in ollama?

Please do not hijack an issue about one subject to ask a question about another subject.

To stay organized issues are automatically closed after 90 days of no activity. If the issue is still relevant please open a new one.

fix(app): restore composer action labels (anomalyco#1068)

Navigation Menu

Repository navigation

Tool use with Ollama models. #1068

Credits:

Pre-requisites

Notes

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Tool use with Ollama models. #1068

Description

Activity

smoush7 commented on Jul 17, 2025

DiedeClaessens commented on Jul 17, 2025

leporel commented on Jul 17, 2025

nutt-y commented on Jul 18, 2025

renich commented on Jul 18, 2025

Credits:

Pre-requisites

Notes

azaeldrm commented on Aug 30, 2025

maxolasersquad commented on Sep 11, 2025

vdmitriyev commented on Sep 26, 2025

BhautikChudasama commented on Oct 5, 2025

rekram1-node commented on Oct 5, 2025

maxolasersquad commented on Oct 5, 2025

thdxr commented on Mar 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions