Local Ollama tool calling either not calling or failing outright

Hi, I'm trying a local Ollama model that supports tool calls (qwen3:32b). When I ask for example "What networks am I connected to?" the model just thinks about which tool to use to fulfill the request, but never does anything. Occasionally it will do something, as I've seen with prompts like "What's in my .config files?", and its seemingly correctly generating the tool-call JSON in the ballpark of {"Bash": "LS", ...} or something, but the command never actually executes anything and is just printed to the chat. I'd assume something isn't configured right or maybe the JSON tool call is not formatted correctly after all. I've confirmed it works with Github Copilot GPT-4o, and all tools were enabled in my settings for both Copilot and Ollama. What is the expected workflow to get opencode to actually run the command after the model suggests it?

(I simplified my config for space here to whats important, but nothing else is different except just some other Ollama models I added.)

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://127.0.0.1:11434/v1"
      },
        "qwen3:32b": {
          "name": "Qwen 3 32B (local)"
      }
    }
  }
}

Same here , but i am using litellm in front of ollama for OpenAI compatibility adding a token

Update with more models below.

TLDR, nothing works except cloud models. With Ollama, some models don't acknowledge having access to tools, some do but only explain what they would do and never actually try to do anything with them, some actually try but get the formatting wrong or are simply unrecognized by opencode, and some try to run tools and appear to be recognized by opencode but still don't do anything.

~~~~~~~~~~ OLLAMA ~~~~~~~~~~

Cogito v1 Preview 32B Q4_K_M (cogito:32b)

Details

# Conversation History

User (2025-07-16 02:27:00)

What networks am I connect to?

Assistant (2025-07-16 02:27:00)

To determine which network your system is connected to, let me first try running some basic commands:

Devstral 24B Q_4_K_M (devstral:24b)

Details

# Conversation History

User (2025-07-16 02:29:44)

What networks am I connected to?

Assistant (2025-07-16 02:29:44)

I'm unable to directly determine your network connections. However, you can typically find this information by checking your device's network settings or using command-line tools like ifconfig on Unix-based systems or ipconfig on Windows.

If you're looking for guidance on how to check this for a specific operating system, let me know!

Qwen 3 32B (qwen3:32b)

Details

# Conversation History

User (2025-07-16 02:33:13)

What networks am I connected to?

Assistant (2025-07-16 02:33:13)

Okay, the user is asking, "What networks am I connected to?" Let me think about how to approach this.

First, I need to figure out which tools are available. The provided tools include functions like Glob, Grep, LS, Read, etc. But the user is asking about network connections, which might not be directly accessible through these tools.

Wait, there's the 'Bash' tool. Maybe I can run a command to check network connections. For example, on a Unix-based system, the 'netstat' command or 'ifconfig' could show network interfaces and connections. Alternatively, 'ip a' or 'nmcli' might be useful.

Since the user is likely working in a Linux environment, using 'bash' to execute a command like 'nmcli dev status' could list the network devices and their status. This command is part of NetworkManager, which is common in many Linux distributions.

But I should check if the Bash tool is available. Looking back at the tools listed, there is a 'task' function that launches an agent with access to Bash. So, maybe using the task function to run a Bash command would be the right approach here.

Alternatively, if the system doesn't have NetworkManager, maybe using 'ip a' or 'ifconfig' would be better. But without knowing the exact environment, 'nmcli' is a good starting point.

So, the plan is to use the 'task' function to execute a Bash command that checks network connections. The prompt for the agent should be to run the appropriate command and return the output.

Therefore, the function call would be to the 'task' function with a description "Check network connections" and a prompt instructing the agent to run the Bash command and return the result.

[Tool: task]

Okay, the user wants me to display the current network interfaces and their status. They mentioned examples like 'nmcli dev status' or 'ip a'. I need to decide which command to use.

First, I should check which tool is more appropriate here. The 'ip a' command is part of the iproute2 suite and shows all network interfaces along with their addresses and status. The 'nmcli dev status' command is specific to NetworkManager and shows the status of devices managed by it. Since the user is asking for a general overview, 'ip a' might be more comprehensive as it doesn't rely on NetworkManager being present.

However, I should consider the environment. If the system uses NetworkManager, then 'nmcli' would provide useful info. But without knowing the system's configuration, I should opt for a command that's more universally available. 'ip a' is part of the standard Linux tools, so it's safer to use that.

So the correct function call is to execute 'ip a' using the Bash execute function. That should display all interfaces and their details.

Qwen2.5 Coder 32B Instruct IQ3_M (krith/qwen2.5-coder-32b-instruct:IQ3_M)

Details

# Conversation History

User (2025-07-16 02:45:50)

What networks am I connected to?

Assistant (2025-07-16 02:45:50)

{"name": "task", "arguments": {"description": "Check network connections", "prompt": "Research and report the current network connections by using system commands like ipconfig (Windows) or ifconfig/ifdata (Linux)."}}

User (2025-07-16 02:46:09)

What is Rust?

Assistant (2025-07-16 02:46:09)

To provide information about Rust without launching an agent or reading specific files, I can summarize the key points about Rust here:

Rust is a systems programming language created by Mozilla Research. It aims to empower all developers to build reliable and efficient software. Rust does this by providing low-level control over hardware resources while also preventing many of the errors that plague other languages at compile time.

Key features of Rust include:

Memory Safety: Rust prevents null pointer dereferences, buffer overflows, and data races through its ownership system.
Concurrency: Rust's ownership model ensures thread safety without heavy runtime overhead, supported by powerful concurrency primitives like threads and async/await.
Performance: Performance is comparable to C++ while offering memory safety, eliminating the need for a garbage collector or manual memory management in most cases.
Toolchain: Rust comes with Cargo, its package manager and build system, which streamlines dependency management and project creation.

If you would like more detailed information about Rust, such as specific features, use cases, or comparisons to other languages, please let me know!

User (2025-07-16 02:46:27)

Whats my username?

Assistant (2025-07-16 02:46:27)

I don't have access to your specific system information or user settings, so I can't provide your username. If you need to find out your username on a particular system, you might consider checking the terminal on Linux or macOS by running whoami, or on Windows by typing echo %username% in Command Prompt.

Is there something else specific you'd like help with?

User (2025-07-16 02:46:38)

What networks am I connected to?

Assistant (2025-07-16 02:46:38)

task {"description": "check network connections", "prompt": "Identify all active network interfaces and their IP addresses on the local machine. Do not write code; just return a concise summary of the network connections."}

~~~~~~~~~~ CLOUD ~~~~~~~~~~

GPT4o via GitHub Copilot (it doesn't say but it ran nmcli, iwconfig, and ifconfig which is great. Local IPs hidden in paranoia.)

Details

# Conversation History

User (2025-07-16 02:39:08)

What networks am I connected to?

Assistant (2025-07-16 02:39:08)

[Tool: bash]

You are connected to the following networks:

Loopback (lo):
- IPv4: 127.0.0.1
- IPv6: ::1
Ethernet (enp7s0):
- IPv4: ###.###.###.###/24 (Dynamic)
- IPv6: ####:####:####:####:####:####:####:#### (Global, Dynamic)
- IPv6: ####::####:####:####:#### (Link-local)
Wi-Fi (wlp8s0):
- IPv4: ###.###.###.###/24 (Dynamic)
- IPv6: ####::####:####:####:#### (Link-local)

Let me know if you need further details or assistance!

Claude Sonnet 3.5 via GitHub Copilot

Details

# Conversation History

User (2025-07-16 02:58:27)

What networks am I connected to?

Assistant (2025-07-16 02:58:27)

[Tool: bash]

You are connected to:

Local network (###.###.###.### via ethernet
Wireless network (###.###.###.###)

local LLMs are not yet ready for this kind of workload

Please explain... in my startup we use all the big models but also we have a big server with devstral, deepseek and qwen that helps a lot to develop... exactly what problems have local LLMs ?

Interesting. Is this an inherent issue with open models, like limited parameter count or with how open models are fine-tuned at the moment for tool calling? Either way, projects like open-interpreter seem to circumvent this, I think by ditching the standard tool calling protocol or something like that. Could open-interpreter be integrated, available as a provider, or possibly be given an openai compatible API and local endpoint of its own just like Ollama has?

In fact we are using ollama but with litellm for openai compatibility.
And we are coding really well with cline or kilocode however i prefer the cli and right now i haven't tested a good cli coding agent Open source. For that we mainly use géminis code assist and Amazon q developer

@p-lemonish

OK. ollama user here in Fedora 42.

I was looking for this solution for a while. This is why I'm posting it here (and all issues) for all to know:

@p-lemonish discovered why and the fix is easy: https://github.com/p-lemonish/ollama-x-opencode

In short, download a model that has tool capabilities:

$ ollama show granite3.3:8b
  Model
    architecture        granite    
    parameters          8.2B       
    context length      131072     
    embedding length    4096       
    quantization        Q4_K_M     

  Capabilities
    completion    
    tools         

  License
    Apache License               
    Version 2.0, January 2004    
    ...                          

And edit it so that it has a fixed num_ctx:

```terminal
$ ollama run qwen3:latest
>>> /set parameter num_ctx 16384
Set parameter 'num_ctx' to '16384'
>>> /save qwen3:latest
Created new model 'qwen3:latest'
>>> /bye

After that, you're good to go. Just try it out:

opencode run "generate a todo.md file with the contents 'hello world, from me' in it" --model ollama/qwen3:latest

With a bit of luck, your file will be there.

Credits:

https://github.com/p-lemonish/ollama-x-opencode

Pre-requisites

Your model must be downloaded and functional in ollama.
You need to configure your model in opencode.json

My configuration looks like this:

{
    "$schema": "https://opencode.ai/config.json",
    "provider": {
        "ollama": {
            "npm": "@ai-sdk/openai-compatible",
            "options": {
                "baseURL": "http://localhost:11434/v1"
            },
            "models": {
                "granite3.3:8b": {
                    "tools": true
                },
                "mistral-small3.2:24b": {
                    "tools": true
                },
                "qwen3:latest": {
                    "tools": true,
                    "reasoning": true
                }
            }
        }
    }
}

Notes

granite3.3 still doesn't work.

thanks for writing that up!

to clarify what i said earlier - what i meant was the majority of models do not have a great out of the box experience for the type of workload that opencode asks from them

calling tools in a loop, being persistent, working around mistakes/errors

sonnet, gpt4.1, and k2 are good at this workload

other models can be made to work by playing with system prompt / parameters but we likely need research like the above post to figure out how to get them to behave well

@renich

@renich
Thanks a lot, it's working for me with qwen3:14b directly from Ollama (not with docker).
I just /set and /save parameter num_ctx 16384

When I use mlx-omni-server to serve Qwen3-Coder, I can easily get it to call tools, but opencode doesn't perform the tool calls:
list files in the current directory
pbrass (10:00 PM)

[lists files in the current directory]

<tool_call> <function=list> <parameter=path> /Users/pbrass/ai/test1 </tool_call>
mlx-community/Qwen3-Coder-480B-A35B-Instruct-4bit (10:00 PM)

The model sends the <tool_call> request, but for some reason it doesn't get executed. I feel like I"m this close to getting it working. I wonder what part of response parsing isn't working.

similar tool call issue for me; it was working a couple weeks ago with devstral, wondering if a new error was introduced

The fix is relatively easy: and described here: https://github.com/p-lemonish/ollama-x-opencode
Short answer is: Your local run does not have a large enough context window.

Two things that are not mentioned:

Your default installation of ollama may only run on the CPU! (Arch installation yay -Sy ollama-gpu, on other systems you may want to run the manual installation of ollama (https://github.com/ollama/ollama/blob/main/docs/linux.md#manual-install)
Get a model that actually is able to run tools! (https://ollama.com/search?c=tools)
Replicate the steps described in https://github.com/p-lemonish/ollama-x-opencode?tab=readme-ov-file#opencode-setup-with-ollama
and save your model that has the num_ctx setting set (above 16k!, check your model how large it can be, and keep performance in mind.)
??
Test and profit.

Jokes aside, the two parts I was missing are: I assumed my local ollama was running on the GPU, it was not. And the larger context window was too much for the CPU to handle. Careful since larger context sizes will be quite demanding.

Just tried on a beefy workstation with devstral and 192k context window and works like a charm (faster than most web based API).
Tried on my home workstations GPU at 16k context window, tokens trickling out slowly, but it works.

For me using a larger context with Qwen3 sort of works, however it fails to edit files intermittently with various errors along the lines of:

Error: oldString not found in content or was found multiple times.

Sometimes it succeeds, and sometimes it is able to recover from errors via its reasoning chains, but still it's something that partially prevents effective use.

I understand that technically the model is probably not generating the correct call, but maybe the tool itself could be made more error-tolerant or use a different paradigm?

When it needs to read or write the entire file, it seems to work just fine.

ollama has been unreliable and we receive a large number of issues from people trying to use itwe recommend using LMStudio as it works decently out of the box with qwen3 coder

LMStudio is not open source though, so it is not really comparable. I suppose it could perhaps be sandboxed and firewalled, but it is still extra effort. Of course I cannot speak for all Ollama users, but one of the reasons to run Ollama is that it is easier to make sure your or your customers' sensitive data is not sent to the cloud.

Do we know that Ollama generates different, unreliable responses using the same Qwen3 models? That also sounds a bit strange.

@vytas7

As a user I do agree with @vytas7 . I am actually one of the many around the internet trying to find a replacement for Claude-Code CLI but not calling claude and call something local. Among the different tools available, definitely opencode is the most mature - but I do struggle with ollama.

I made it work with LM Studio - but was kind of slow - maybe is my config.

By the way qwen3-coder seems not to support tools :(

qwen3-coder:latest

{ "$schema": "https://opencode.ai/config.json", "provider": { "ollama": { "npm": "@ai-sdk/openai-compatible", "options": { "baseURL": "http://localhost:11434/v1" },qwen3-coder:latest "models": { "qwen3-coder": { "tools": true } } } } }

@javapapo

@javapapo Qwen3-coder was specifically designed for agentic tools. If ollama says it's not usable then it's wrong. Try to use tools after removing that line of yaml ""tools": true". Otherwise it's time to switch to llama.cpp

fair thanks for the reply.

FYI, The first releases of Qwen3-coder had metadata wrong and tools were not in the metadata. That's why a number of other versions of this popped up addressing it. I used those versions (and even created my own), but I have almost no luck getting opencode to reliably get models to make tool calls via ollama. Also used llama.cpp and lmstudio with very little luck. I dearly want to use/love opencode, but I haven't had much success. For reference, I have a 24gb 5090 so I'm not just trying out the small parameter models, and I also bump up the default 4096 context, but still no luck.

…

On Thu, Aug 28, 2025 at 8:37 AM lemonish ***@***.***> wrote: *p-lemonish* left a comment (anomalyco/opencode#1034) <#1034 (comment)> @javapapo <https://github.com/javapapo> Qwen3-coder was specifically designed for agentic tools. If ollama says it's not usable then it's wrong. Try to use tools after removing that line of yaml ""tools": true". Otherwise it's time to switch to llama.cpp — Reply to this email directly, view it on GitHub <#1034 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQLVQW73KZWGZDNX7LP2L3P3ZW5AVCNFSM6AAAAACBTOORFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMZTGMZTEOJZGA> . You are receiving this because you commented.Message ID: ***@***.***>

FYI, The first releases of Qwen3-coder had metadata wrong and tools were
not in the metadata. That's why a number of other versions of this popped
up addressing it.

I had this issue too, but updating to the newest version helped immediately. I have managed to get the basic agentic tools working without issues. For anything specific I have created my own MCP with python and had the model use that tool through it. I didn't try using the MCP for a couple months now, but it worked fine before.

For general MCP from various providers off the internet I feel was too "chaotic" or not standardized enough to work well or at all.

Interesting that with llama.cpp it's not working for you. You can probably use llama logs to debug the issue.

To be clear, you updated to the latest version of what? Opencode, Qwen3 Coder, Ollama, llama.cpp? I'll give it another run once I hear what your working setup is. Thanks!

…

On Fri, Aug 29, 2025 at 4:57 AM lemonish ***@***.***> wrote: *p-lemonish* left a comment (anomalyco/opencode#1034) <#1034 (comment)> FYI, The first releases of Qwen3-coder had metadata wrong and tools were not in the metadata. That's why a number of other versions of this popped up addressing it. I had this issue too, but updating to the newest version helped immediately. I have managed to get the basic agentic tools working without issues. For anything specific I have created my own MCP with python and had the model use that tool through it. I didn't try using the MCP for a couple months now, but it worked fine before. For general MCP from various providers off the internet I feel was too "chaotic" or not standardized enough to work well or at all. Interesting that with llama.cpp it's not working for you. You can probably use llama logs to debug the issue. — Reply to this email directly, view it on GitHub <#1034 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQLVUA463OCTGBBGV5QA33QAIY3AVCNFSM6AAAAACBTOORFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEMZWGI3TCMJSHA> . You are receiving this because you commented.Message ID: ***@***.***>

To be clear, you updated to the latest version of what? Opencode, Qwen3
Coder, Ollama, llama.cpp?

Sorry, wasn't clear enough. I had an out of date Qwen3-coder, the first one that came out, but didn't use it since. Started using it again and found out that it didn't work, but it gave quite specific error messages about not finding tools or understanding it; I found people were talking about broken metadata for qwen3-coder and were prompting all users to update to the newest version of qwen3-coder. I updated to the newest version from unsloth some days ago and it has worked now.

llama.cpp version: build: 5827 (67d1ef23) with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu

Didn't update that for a while either. For opencode it seems to work with any given version I've been on, it's auto-updating for me.

[automated] Closing due to 90+ days of inactivity. Feel free to reopen if you still need this!

feat: add hexagonal architecture SKILL. (anomalyco#1034)

fix(desktop): persist worker share tokens across restarts (anomalyco#…

/reopen

@renich

@renich's solution still works for me to get qwen3.5:9b working with opencode. Thanks!

I just wanted to add that the current recommendation from ollama docs is to set the context length to 64K tokens instead:

Tasks which require large context like web search, agents, and coding tools should be set to at least 64000 tokens.

$ ollama run qwen3.5:9b  

/set parameter num_ctx 65536
/save qwen3.5:9b

With the new, much more powerful, local model families such as Qwen 3.8 or Muse Glimmer, this is resolved at least for me. Use at least 128k context (or maybe 64k is enough for very small projects).

However, it would be nice of OpenCode allowed to reduce the number of available tools and effectively the size of system prompt; I noticed pi.dev performs somewhat better with the same models.

Navigation Menu

Repository navigation

Local Ollama tool calling either not calling or failing outright #1034

Credits:

Pre-requisites

Notes

8 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Local Ollama tool calling either not calling or failing outright #1034

Description

Activity

alfonsodg commented on Jul 16, 2025

smoush7 commented on Jul 16, 2025

thdxr commented on Jul 16, 2025

alfonsodg commented on Jul 16, 2025

smoush7 commented on Jul 16, 2025

alfonsodg commented on Jul 16, 2025

renich commented on Jul 18, 2025

Credits:

Pre-requisites

Notes

thdxr commented on Jul 18, 2025

fantomac commented on Jul 23, 2025

pbrass commented on Jul 30, 2025

ggaabe commented on Jul 30, 2025

maiterth commented on Jul 30, 2025

vytas7 commented on Aug 3, 2025

8 remaining items

vytas7 commented on Aug 27, 2025

javapapo commented on Aug 27, 2025

p-lemonish commented on Aug 28, 2025

javapapo commented on Aug 28, 2025

zendril commented on Aug 28, 2025

p-lemonish commented on Aug 29, 2025

zendril commented on Aug 29, 2025

p-lemonish commented on Aug 29, 2025

rekram1-node commented on Dec 28, 2025

Jeeppler commented on Jul 20, 2026

sleeyax commented on Aug 23, 2026

vytas7 commented on Aug 24, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions