Skip to content

[FEATURE]: Code Mode needs a token budget and a per-tool breakdown #58708

Open

Description

@RicSchonfelder

I measured on my own machine before asking for anything. OpenCode v2.1.4, Linux, one MCP server with 207 tools.

rea is one MCP server I have configured (a reverse-engineering toolkit). Its tools/list response has 207 tools. Dumped as JSON and counted with tiktoken o200k_base, those definitions are 3,423,785 characters and 1,033,360 tokens. One tool, address_name, is 2,721 tokens by itself.

Traditional tool calling injects every schema into every request, so this single server would spend 1.03M tokens before I type anything. That fits no 200k window and eats a 1M window whole. With search(), the tool, the system prompt and the two signatures the task actually needed came to about 6.6k tokens.

Then I ran the same task twice in two real sessions on this machine. Fetch 8 URLs, report which ones fail or mention "token". Session A used one tool call per URL and returned each body whole. Session B ran one program that returned a filtered array. Same model, same day, billing read from session_v2.

A: one call per URL B: one program
tokens in 31,427 696
tokens out 1,566 505
reasoning 1,496 56
cache read 426,752 44,544
total tokens 34,489 1,257
context processed 458,179 45,240
cost $0.00783 $0.00057

27.4x fewer tokens, 13.6x cheaper. The dollar gap is smaller than the token gap because the repeated prefix hits prompt cache.

A also got 3 of the 8 answers wrong. Its bodies were truncated at 50k characters, which cut off the word "token" on three pages. B searched the full text inside the sandbox and got all 8 right. Truncating observations changed three answers and the price.

A synthetic run of the same task counts 114,320 tokens for one call per URL against 2,436 for one program, 46.9x apart, and 24.5x apart once the prefix is priced at a 10% cache rate. That cost grows with N² because every observation is resent in every later request: at the 2,752-token average, 20 URLs cost 578k tokens and 50 cost 3.5M.

None of this is news to Cloudflare or Anthropic. Kenton Varda and Sunil Pai published the same approach in September 2025 (https://blog.cloudflare.com/code-mode/), and Anthropic shipped programmatic tool calling in November 2025 with a reported 37% average drop on research tasks. OpenCode already ships the mechanism; a default policy and a per-tool breakdown are missing.

I read #8625 (MCP search, shipped via #12520) and #53121 (per-step cost, open) before writing this. Both sit below what I am asking for.

What I would like to see, and the questions behind each item

1. A default for large MCP servers. Above roughly 20 tools, should search + execute become the default exposure instead of every schema? Right now a user has to know this, and the fallback degrades any model with a smaller window. Would mcp.servers.<name>.mode: "search" | "direct" fit the config, or is there a reason to keep schemas inline?

2. Per-tool and per-schema tokens. #53121 asks for per-step and per-turn cost. I want the split inside one step: system prompt, tool schemas, each tool observation, reasoning, output. Without it I cannot tell whether a session burned 40k on a page fetch or on 180 unused schemas. Is that already stored per message, and would a TUI view or an opencode api field be acceptable?

3. Output caps that spill instead of truncate. shell and webfetch already keep the full output in managed storage. read pages at 50 KiB and MCP results come back whole. Could a configurable cap (say 4k tokens per tool result, full output in a file, path returned) become the default for every tool, MCP included?

4. Does execute reach built-in tools? #57720 and #57627 read like this is still settling. If execute could call read, grep, edit and shell as functions, a six-step file task becomes one inference pass. Is that planned, or does Code Mode stay MCP-only on purpose?

5. Compaction accounting. One Code Mode step can hide 50 underlying calls. Does compaction weigh it as one message when it decides what to keep? If yes, a summary can throw away everything the program found. If no, large code steps can exceed compaction.keep.tokens on their own.

6. Sandbox and audit. My execute has fetch but no require('fs') (#56811). What are the actual limits on network, filesystem, memory and time, and can a user inspect or replay the code that ran? This decides how much I trust it as the default path.

What I would like from the community

If you run an MCP server with a lot of tools, please post three numbers: how many tools it exposes, what its tools/list payload costs in tokens, and your session cost before and after switching to search + execute. MCP servers with 50+ tools are the interesting ones.

I can turn the benchmark into opencode mcp cost <name> if you want it in the repo, or ship it as a plugin if core is not the place for it.

Benchmark, raw numbers and both measurement scripts: https://gist.github.com/RicSchonfelder/268a51cb86395db36eaac0aa018686cd

Activity

  1. opencode-agent commented on Oct 24, 2026

    @opencode-agent
    Contributor

    Thanks for measuring all of this and sharing the numbers and scripts.

    In short, you're asking for a default token policy for Code Mode and a per-part view of where tokens go. Maintainers will consider it. Some notes on each item, from reading the current dev code:

    The compaction accounting (5) and the per-part split within one step (2) don't seem to be tracked anywhere yet. Separate, focused issues for those would make them easier to act on.

  2. RicSchonfelder commented on Oct 24, 2026

    @RicSchonfelder
    Author

    No bug on codemode: the 1.03M figure is the tools/list payload I got by talking to rea mcp over stdio directly, outside OpenCode. Inside 2.1.4 my context only ever held 2 or 3 rea tools at a time, which is exactly what Code Mode is supposed to do. I should have said that in the first post instead of "before I type anything".

    So item 1 shrinks to what you described: a threshold. Two things still look worth doing, and neither is about the default:

    The payload still matters to the server's other clients. I run the same MCP from OpenCode, Claude Code and Cline. Only OpenCode lazy-loads. For the rest, rea's default catalog is 1,033,358 tokens and the request is rejected before a single tool call. The interesting part is that the fix was available on the server side: REA_MCP_OUTPUT_SCHEMA_PROFILE=off plus REA_MCP_INPUT_SCHEMA_PROFILE=compact takes that same catalog to 56,949 tokens (5.5%), and a client-side truncating proxy to 21,130 (2.0%). outputSchema alone was 75% of the payload. If search is the default, a tool-count threshold may not be needed on the client at all, but a note in the MCP docs ("if your server ships output schemas, that is what your client is paying for") would have saved me a day.

    search quality depends on what the catalog advertises. With 207 tools and one-line descriptions, matching "compare two web captures" to compare_web_captures is fine. It gets worse when descriptions are truncated by a proxy or by the server's compact profile, and there is no way to see whether search returned the right tool before spending a step on it. If a threshold lands, it would help to keep the full description text for the searchable catalog and only trim what gets injected.

    Measurements behind all of this, on my machine, in case they help calibrate: 16,910 requests in seven days, average context 126,169 tokens, 2.13B cached tokens against 92.9M fresh. The cached context is where the money goes, which is why I care so much about what the prefix costs.

    Per your suggestion I will open the two untracked items separately: the per-part split inside a step, and compaction accounting for Code Mode steps. I will link them here so this thread can stay about the default policy and the threshold.

  3. RicSchonfelder commented on Oct 24, 2026

    @RicSchonfelder
    Author

    Opened as you suggested: #58716 (per-part split inside a step) and #58717 (compaction accounting and keeping a Code Mode step's return value).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions