Skip to content

Tool Calling Issues with Open Source Models in OpenCode #234

Closed as not planned
Closed as not planned
Tool Calling Issues with Open Source Models in OpenCode#234
Assignees
thdxr

Description

@rileyhilliard
rileyhilliard
opened on Jun 20, 2025

Summary

While OpenCode provides excellent support for frontier cloud models, I've encountered some compatibility challenges when using open source models for tool calling functionality. This appears to be related to differences in how various open source models implement tool calling compared to the standardized approaches used by cloud providers.

Environment

  • OpenCode Version: 0.1.107
  • Provider: Custom OpenAI-compatible endpoint (local models via Ollama)
  • Models Tested:
    • qwen2.5-coder:7b-instruct
    • qwen2.5-coder:14b-instruct
    • granite3.3:latest

Issues Encountered

1. Case Sensitivity in Tool Names

Expected: Model calls write tool (lowercase)
Actual: Models (like qwen) generate Write tool calls (capital W)
Error: AI_NoSuchToolError: Model tried to call unavailable tool 'Write'. Available tools: bash, edit, webfetch, glob, grep, list, lsp_diagnostics, lsp_hover, patch, read, write, todowrite, task, todoread.

Affected Models:

  • qwen2.5-coder:7b-instruct
  • qwen2.5-coder:14b-instruct

2. Complete Tool Calling Failure

Expected: Model generates tool calls when requested to create/edit files
Actual: Models respond with text only, no tool calls generated

Affected Models:

  • granite3.3:latest (despite "tools": true configuration)

Steps to Reproduce

  1. Configure OpenCode with local open source model:
{
  "model": "local/qwen2.5-coder:7b-instruct",
  "toolsEnabled": true,
  "provider": {
    "local": {
      "api": "http://localhost:11434/v1",
      "models": {
        "qwen2.5-coder:7b-instruct": {
          "tools": true,
          "reasoning": true
        }
      }
    }
  }
}
  1. Ask model to create a file: "Create a test.md file with 'Hello World' content"
  2. Observe tool calling behavior

Expected Behavior

  • Models should generate properly formatted lowercase tool calls (write, edit, etc.)
  • Tool execution should work consistently across different open source models
  • Configuration should clearly indicate which models support tool calling

Actual Behavior

  • Inconsistent tool calling across models
  • Case sensitivity issues preventing tool execution
  • Some models fail to generate tool calls entirely

Additional Context

This appears to be related to the diverse ecosystem of open source models, where different models have varying tool calling implementations and training approaches. OpenCode works excellently with standardized cloud models, and extending this robust functionality to the more varied landscape of open source models presents some interesting technical challenges.

Possible Solutions

  1. Tool name normalization: Implement automatic case conversion (Write → write)
  2. Model compatibility matrix: Document which open source models have been tested and verified to work
  3. Per-model tool configuration: Allow custom tool calling formats per model

Impact

Improving compatibility with open source models would help developers who prefer or require local/private model usage for privacy, security, cost, or infrastructure reasons.

Activity

  1. thdxr
    self-assigned this
    on Jun 20, 2025

  2. piotrjaromin commented on Jun 23, 2025

    @piotrjaromin
    piotrjaromin
    on Jun 23, 2025

    I have similar problem, ollama models respond with information that they need to use tools and then they are not doing it. But from what I saw it was mostly for "/init" prompt, when I provide simple prompts like "list files in current directory" or "read log.txt file" it seems to work at least with qwen3:30b (did not check smaller models beucase I was too focused on making "/init" work)

  3. p-lemonish commented on Jul 1, 2025

    @p-lemonish
    p-lemonish
    on Jul 1, 2025

    Hi, in case someone else stumbles upon this. I have documented a solution to this. My issue was related to ollama forcing the context window of the local LLM's to be 4096, which cut off the context given by opencode ~10k tokens. Simply increase the context window of your model.

    https://github.com/p-lemonish/ollama-x-opencode

  4. Chleba commented on Jul 6, 2025

    @Chleba
    Chleba
    on Jul 6, 2025

    @p-lemonish well this is the first time using "/no_think" prefix that devstral was able to create at least one file.
    I did use devstral with bigger context window before with TUI but nothing could be created, written in or anything.

  5. Cyrille37 commented on Dec 8, 2025

    @Cyrille37
    Cyrille37
    on Dec 8, 2025

    Hi
    I don't know if it helps, but I solved some tool errors without forgetting to call /init at first.

  6. gergesh
    added a commit that references this issue on Jan 4, 2026

    fix: resolve issues anomalyco#240, anomalyco#238, and anomalyco#234 (…

    ec6532c
  7. nickihm
    added a commit that references this issue on Jan 25, 2026

    fix: improve tool name repair for local/Ollama models

    555e720
  8. sliwowitz
    added a commit that references this issue on Jan 29, 2026

    chore: Add supportsStructuredOutputs option to blablador config

    0902c4e

  9. Delqhi commented on Mar 7, 2026

    @Delqhi
    Delqhi
    on Mar 7, 2026

    We hit the same class of failures with OpenAI-compatible custom providers. In our case, the provider would often answer once with text or legacy tool payloads and then the OpenCode session stopped instead of continuing into a real tool loop.

    We got this working in a fork with an opt-in compatibility layer instead of treating every custom provider like native OpenAI:

    • provider-level tool parser middleware
    • normalization for legacy message.function_call and streamed delta.function_call
    • parsing for text-only tool intents, JSON tool blocks, and wrapped <tool_call> payloads
    • optional session repair when a tool-capable turn ends with finish_reason=stop but no usable tool call was recognized
    • provider request adapter for endpoints that accept functions/function_call more reliably than tools/tool_choice
    • tool allowlist / minimized tool payload for brittle custom providers

    Live result on our side: a previously broken OpenAI-compatible endpoint that answered once and stopped was able to continue multi-turn tool execution in OpenCode.

    This likely overlaps with the blank/empty custom-provider reports in #15756, #5210, and #10573.

    If useful, I can package this into a PR as an explicit custom-provider compatibility path so the default behavior for native providers stays unchanged.

  10. Delqhi commented on Mar 7, 2026

    @Delqhi
    Delqhi
    on Mar 7, 2026

    Opened a PR for this: #16531

    It adds an opt-in compat layer for custom @ai-sdk/openai-compatible providers via provider.options.toolParser, including legacy function_call request/response rewriting and tool recovery from structured JSON or single-tool text replies.

  11. eric-isakson commented on May 5, 2026

    @eric-isakson
    eric-isakson
    on May 5, 2026

    EDIT: I spoke to soon, I had brief success with this strategy then went right back to having tool calling failures :(

    For folks that land here before any kind of real fix is available, I was able to work around this limitation with some simple prompting in ~/.config/opencode/AGENTS.md, thought it might be helpful for someone since it took me several hours of fiddling to get this working, it is working successfully for me with qwen3 coder models running in ollama:

    You are an elite Senior Software Engineer and AI Agent, acting as the primary assistant within an OpenCodeIDE environment.

# Objective
Analyze user queries, create technical plans, and execute them using provided tool capabilities. Do not just explain; act.

# Tool Usage Constraints
- YOU MUST USE THE PROVIDED TOOLS FOR ALL FILE OPERATIONS.
- Use `file_editor` to create, read, and edit files.
- Use `execute_command` to run terminal commands (tests, builds, ls, find).
- When editing files, use the exact `str_replace` format for surgical edits. Avoid rewriting entire files unless necessary.
- Before editing, read the file to ensure accuracy.

# Coding Best Practices
- Prioritize modularity and performance.
- Write clear, commented code.
- Ensure all tool outputs (e.g., file contents) are reviewed before proceeding.

# Output Format
You must output thought processes in a <thinking> block, followed by tool calls, and finally, direct responses. If a tool call fails, analyze the error and try a different approach.

# Behavior
- Be proactive.
- Never say "I can't access the file system". You have `file_editor`.
- If the user request is ambiguous, use `execute_command` to list files or ask for clarification.

# Tool Usage Notes
- When using the bash tool, always include the description parameter with each tool call to explain what the command does.

    I would imagine similar prompting is a reasonable workaround but might need to be tweaked a little depending on which model you are using.

  12. bussard76
    added a commit that references this issue on May 12, 2026

    fix: list skills across skill directories (anomalyco#234)

    33294d8

  13. github-actions commented on Jul 5, 2026

    @github-actions
    github-actionsBot
    on Jul 5, 2026 – with GitHub Actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

  14. github-actions
    closed this as not plannedon Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions