Skip to content

GPT-5.6 Code Mode omits tool_search from exec, degrading deferred MCP discovery #32101

Open

Description

@An-jinu

Summary

GPT-5.6 model metadata selects code_mode_only while supporting deferred tool search. The Code Mode converter currently drops ToolSpec::ToolSearch, so the public exec tool does not expose the dedicated BM25 tools.tool_search(...) discovery primitive.

Deferred tools are still reachable through manual ALL_TOOLS filtering, but GPT-5.6 loses the ranked discovery path that GPT-5.5 Direct mode retains. In practice this reduces proactive selection of specialized deferred MCP tools such as CodeGraph and encourages fallback to generic shell search.

Environment

  • Codex CLI: codex-cli 0.144.0-alpha.4
  • Upstream Codex checked: 656a2d0905c9e0b9bdade1badab07ef6d42ca17c
  • OS: macOS 26.3.1, arm64
  • Model: GPT-5.6 (tool_mode = code_mode_only, supports_search_tool = true)
  • Comparison model: GPT-5.5 (tool_mode = null, Direct mode)

Repository Decision

  • Target repository: openai/codex
  • Why this belongs here: the omission is in the upstream ToolSpec to Code Mode conversion and reproduces independently of the MCP provider.
  • Upstream evidence: current codex-rs/tools/src/code_mode.rs converts ToolSpec::ToolSearch { .. } | ToolSpec::WebSearch { .. } to Vec::new().

Reproduction

  1. Use a model with tool_mode = code_mode_only and supports_search_tool = true (current GPT-5.6 metadata), with at least one searchable deferred MCP namespace.

  2. Start a turn and inspect the nested declarations in the public exec tool.

  3. Observe that deferred MCP metadata is available through ALL_TOOLS, but there is no tools.tool_search(...) declaration.

  4. The converter behavior is also locked by the current upstream unit test:

    cargo test -p codex-tools \
      tool_spec_to_code_mode_tool_definition_skips_unsupported_variants \
      -- --nocapture

    It passes by asserting that converting ToolSpec::ToolSearch returns None.

Expected Behavior

When search support causes MCP tools to be deferred, Code Mode should expose tool_search as a nested function inside exec. Individual deferred MCP schemas should remain absent from the initial prompt.

Actual Behavior

The planner correctly creates and registers ToolSearchHandler, but both the Code Mode prompt-definition and runtime-definition paths pass the spec through collect_code_mode_*_tool_definitions. ToolSpec::ToolSearch is discarded there, leaving only the less capable instruction to manually filter ALL_TOOLS by name and description.

Evidence

  • codex-rs/models-manager/models.json: GPT-5.6 Sol/Terra/Luna use code_mode_only with search support; GPT-5.5 does not force Code Mode.
  • codex-rs/core/src/mcp_tool_exposure.rs: searchable MCP tools are intentionally deferred.
  • codex-rs/core/src/tools/spec_plan.rs: the deferred runtimes and ToolSearchHandler are registered before Code Mode conversion.
  • codex-rs/tools/src/code_mode.rs: ToolSpec::ToolSearch is converted to an empty definition list.
  • codex-rs/core/src/tools/code_mode/execute_handler.rs: the same conversion determines which nested tools are executable in V8.
  • Existing integration coverage confirms deferred tools can still be called using ALL_TOOLS.find(...); the defect is specifically the missing ranked search primitive, not total tool unreachability.

Local live A/B evidence: before the patch, a GPT-5.6 repository-graph lookup fell back to rg without invoking CodeGraph. With the patch, the same path invoked codegraph_explore first and completed without an rg fallback.

Related but distinct issues:

Root Cause

The deferred-tool planner assumes the search primitive will remain available when it defers MCP schemas. That assumption is true in Direct mode, but Code Mode treats the hosted ToolSpec::ToolSearch variant as unsupported and removes it from both the generated exec guide and the V8 runtime catalog.

Two adjacent bridges are also needed once the definition is preserved: Code Mode invokes nested functions as ToolPayload::Function, while ToolSearchHandler originally accepts only ToolPayload::ToolSearch; and the search result needs a Code Mode result shape containing the serialized LoadableToolSpec[].

Proposed Fix

  1. Convert ToolSpec::ToolSearch into a Code Mode function definition using its existing description and parameter schema.
  2. Accept the Code Mode ToolPayload::Function form in ToolSearchHandler and parse it as SearchToolCallParams, while preserving the native hosted-tool path.
  3. Return the serialized LoadableToolSpec[] through ToolOutput::code_mode_result.
  4. Keep hosted WebSearch behavior unchanged and keep individual deferred MCP schemas out of the initial exec description.

Reference implementation:

I am sharing the commit as implementation context rather than opening an unsolicited PR. If this approach aligns with the intended architecture, I would be happy to open an invited PR, or the commit can be cherry-picked directly.

Verification Plan

  • Unit: ToolSpec::ToolSearch converts to an augmented Code Mode function definition.
  • Output: Code Mode receives a JSON array of loadable tool specs while the native ToolSearchOutput response remains unchanged.
  • Integration: in CodeModeOnly, exec documents tools.tool_search(...), search returns a non-empty result, and the still-deferred MCP tool remains callable without leaking its schema into the initial prompt.
  • Live: repeat a GPT-5.6 graph-shaped repository query and verify CodeGraph is selected without an rg fallback.

Local verification on the reference commit:

cargo test -p codex-tools tool_spec_to_code_mode_tool_definition_supports_tool_search
1 passed; 0 failed

RUST_MIN_STACK=8388608 cargo test -p codex-core --test all code_mode_only_guides_all_tools_search_and_calls_deferred_app_tools
1 passed; 0 failed

Activity

  1. added
    bugSomething isn't working
    CLIIssues related to the Codex CLI
    mcpIssues related to the use of model context protocol (MCP) servers
    tool-callsIssues related to tool calling
    on Jul 10, 2026
  2. ignatremizov commented on Jul 11, 2026

    @ignatremizov

    @An-jinu I reviewed the reference implementation against the current Code Mode and tool runtime paths. The root-cause analysis looks correct, and this is distinct from the adjacent Responses Lite fix in ignatremizov/codex@00f642f7e0: that commit keeps top-level tool_search reachable for deferred V1 multi-agent tools in Responses Lite, whereas this issue concerns preserving ToolSpec::ToolSearch through the nested Code Mode exec conversion and bridging its ToolPayload::Function invocation. Both fixes can coexist.

    Two areas in the reference patch may be worth strengthening:

    1. ToolSearchOutput::code_mode_result currently turns a serialization failure into a successful JSON string. A structured error path, or an explicitly justified infallible serialization expectation, would avoid presenting an error message as a valid search result.
    2. The integration test proves that tool_search returns a non-empty result and that the deferred MCP tool remains callable, but it still selects the callable tool independently through ALL_TOOLS.find(...). A stronger end-to-end assertion would read the tool name from the actual tool_search result and invoke that returned tool, proving the full discovery-to-execution path.

    Neither point changes the validity of the report; they would mainly tighten the implementation contract and regression coverage.

  3. zxsleebu commented on Jul 19, 2026

    @zxsleebu

    i have the same issue. gpt 5.5 works flawlessly. gpt 5.6 doesn't detect my custom built mcp tools

  4. srobinson commented on Aug 6, 2026

    @srobinson

    Confirmed on codex-cli 0.146.0: configured MCP routing fails across the GPT-5.6 Code Mode family (observed on Sol, Terra, and Luna; models.json also marks Sol-WM code_mode_only) because this release removes the MCP deferral opt-out while Code Mode drops ToolSearch; enumerating exact namespaces in features.code_mode.direct_only_tool_namespaces restores current tools, but * and mcp__* are unsupported, so this is only a brittle workaround.

  5. reedlaw commented on Aug 28, 2026

    @reedlaw

    Confirmed on codex-cli 0.150.1. GPT-5.6 Sol failed to discover a connected seven-tool Tidewave server despite explicit AGENTS.md instructions, while manual ALL_TOOLS filtering found and successfully invoked it.

  6. dergachoff commented on Sep 3, 2026

    @dergachoff

    Fresh Codex Desktop datapoint: I explicitly asked to use Safari MCP, but the model used generic CUA instead. After I prompted “deferred tool maybe?”, it checked ALL_TOOLS, found 27 working mcp__safari__* tools, and connected successfully.

    Seems like a clear real-world example of the missing tool_search problem described here.

  7. leo-chorus-one commented on Sep 3, 2026

    @leo-chorus-one

    Confirmed on Codex CLI 0.153.0, Fedora 43 x86_64.

    I asked whether a Prometheus alert condition was currently true. Despite an
    AGENTS.md instruction to prefer MCP connectors, Codex claimed no Prometheus
    connector was available and attempted curl.

    After I mentioned MCP explicitly, Codex searched ALL_TOOLS and immediately
    found the working mcp__prometheus__* tools.

    This matches #32101: deferred tools are callable, but the missing tool_search
    path causes Codex to overlook them and fall back to generic tools. Configured
    MCP servers should work without namespace pinning or knowledge of ALL_TOOLS.

  8. Perdolique commented on Sep 4, 2026

    @Perdolique

    0.153.2. Codex App can't discover half of my MCPs (I have just 6 of them) and their tools.

  9. JDeun commented on Oct 2, 2026

    @JDeun

    SchemaRouter may be useful as an external comparison for this Code Mode exposure gap. I maintain https://github.com/JDeun/SchemaRouter, which can retrieve typed provider/endpoint/field contracts without changing the host's execution authority.

    A useful frozen matrix would compare GPT-5.5 Direct/native BM25, GPT-5.6 Code Mode/manual ALL_TOOLS discovery, and typed field-aware preselection over the identical deferred catalog. Metrics: required-tool/field recall, specialized-vs-generic fallback rate, unsupported rejection, discovery/context bytes, tool-search round trips and latency. The important contract is that Code Mode exposure differences must not be mistaken for provider unavailability. Negative results reported unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIbugSomething isn't workingmcpIssues related to the use of model context protocol (MCP) serverstool-callsIssues related to tool calling

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions