Skip to content

Read tool fails to read image files even with vision-capable models (paste works) #11306

Closed

Description

@deleon626

Description

When using a vision-capable model through OpenCode, the Read tool fails to read image files from disk, reporting that the model cannot read images. However, pasting the same image directly into the chat works perfectly.

Steps to Reproduce

  1. Configure a vision-capable model (e.g., Gemini 3 Pro, Claude Sonnet 4.5) with proper modalities:
    "modalities": { "input": ["text", "image", "pdf"], "output": ["text"] }
  2. Have an image file on disk (e.g., screenshot.png)
  3. Ask the model to read the image file using the Read tool
  4. Observe error: "Failed to read image: [path], model may not be able to read images"
  5. Paste the same image directly into chat - it works fine

Expected Behavior

The Read tool should recognize that the model supports vision input (via modalities config) and successfully read image files, just like pasted images work.

Actual Behavior

  • Pasted/dragged images: Works
  • @image.png syntax: Works
  • read(image.png) via Read tool: Fails with model capability error

Configuration

Models already have correct modalities configured:

"antigravity-gemini-3-pro": {
  "modalities": { "input": ["text", "image", "pdf"], "output": ["text"] }
}

Related Issues

Environment

  • OpenCode version: Current
  • OS: macOS
  • Models affected: All vision-capable models (Gemini, Claude, etc.)
  • Modalities: Properly configured with "input": ["text", "image", "pdf"]

Root Cause Analysis

The Read tool appears to:

  1. Read the image file as binary/text
  2. Check if model supports vision
  3. Fail during the handoff between file reading and vision processing

Meanwhile, pasted images bypass the Read tool entirely and are sent as API attachments directly.

Suggested Fix

Either:

  1. Fix Read tool to properly handle image files when model has vision modality
  2. Route image file extensions (.png, .jpg, .jpeg, .gif, .webp) directly to vision/attachment path instead of through Read tool

Activity

  1. github-actions commented on Jan 30, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    Feel free to ignore if your specific case differs or involves additional context.

  2. OpeOginni commented on Jan 30, 2026

    @OpeOginni
    Contributor

    Hey @deleon626 does this nly happen while using antigravity as your model provider?

  3. deleon626 commented on Jan 31, 2026

    @deleon626
    Author

    Hey @OpeOginni, apologies for the confusion - I wasn't using the antigravity provider. I'm experiencing this issue with Kimi K2.5 through the MoonshotAI provider via OpenRouter.

    Configuration

    "moonshotai/kimi-k2.5": {
      "name": "Kimi K2.5 (Moonshot AI)",
      "limit": { "context": 262144, "output": 262144 },
      "modalities": { "input": ["text", "image"], "output": ["text"] },
      "options": {
        "provider": {
          "order": ["moonshotai", "fireworks"],
          "allow_fallbacks": false
        }
      }
    }

    Environment

    • OpenCode version: 1.1.40
    • OS: macOS
    • Provider: openrouter → moonshotai/kimi-k2.5

    Reproduction Steps

    1. Start a session with moonshotai/kimi-k2.5 model
    2. Reference an image using @path/to/image.png → Works fine
    3. Ask the agent to "read more images in that folder"
    4. Agent autonomously uses the Read tool to access images → Blows out context window

    The Issue

    Method Result
    Manual copy-paste ✅ Works
    @path/to/image.png explicit reference ✅ Works
    Agent autonomously calls Read tool on image path ❌ Blows out context window

    The problem only occurs when the agent autonomously decides to read an image - not when I explicitly reference it with @ or paste directly. This suggests the Read tool may be encoding/sending images differently than the explicit @ reference mechanism.

  4. deleon626 commented on Jan 31, 2026

    @deleon626
    Author

    Potential Root Cause Identified

    I conducted an exhaustive investigation using parallel subagents and sequential thinking, and I believe I've identified the root cause of this issue.

    Summary

    Root Cause: Tool output attachments are converted to a non-standard type: "media" format that bypasses modality validation and causes providers to incorrectly tokenize base64 image data as text, resulting in 500-600x context window explosion.

    Bug Location

    File: packages/opencode/src/session/message-v2.ts:464

    The toModelOutput() function unnecessarily converts type: "file" attachments to type: "media":

    // Current (BUGGY) code:
    ...attachments.map((attachment) => ({
      type: "media",      // ❌ WRONG - creates non-standard type
      mediaType: attachment.mime,
      data: attachment.url.slice(commaIndex + 1),  // Strips prefix
    }))

    Why This Causes Context Blowout

    1. Modality Bypass: unsupportedParts() in transform.ts:213 only checks type: "file" and type: "image", not type: "media" - so images bypass vision capability validation
    2. Format Mismatch: OpenRouter expects type: "image_url" but receives type: "media"
    3. Provider Fallback: Unknown "media" type triggers JSON.stringify fallback
    4. Text Tokenization: Base64 image data (e.g., 10KB = 13,333 chars) is tokenized as text (~13,333 tokens) instead of as image (~1,500-2,000 tokens)

    Why Paste/@ Works

    Both use type: "file" which:

    • Passes through modality check in transform.ts:unsupportedParts()
    • Gets converted to proper type: "image_url" format by existing conversion pipeline
    • Provider recognizes format correctly

    Proposed Fix

    Preserve type: "file" instead of converting to type: "media":

    // AFTER (FIXED):
    ...attachments.map((attachment) => ({
      type: "file",              // ← Preserve standard type
      mime: attachment.mime,       // ← Keep mime
      url: attachment.url,         // ← Keep full data URL with prefix
      filename: attachment.filename || "image"  // ← Add filename
    }))

    This single change:

    • ✅ Reuses existing modality check
    • ✅ Reuses existing conversion pipeline
    • ✅ Prevents context blowout
    • ✅ Works for all providers

    I've written a comprehensive analysis report with more details: Desktop Report

    Would you like me to open a PR with this fix?

  5. zuojianghua commented on Feb 3, 2026

    @zuojianghua

    same issue at v1.1.48. it read image binary data. then llm context oversized

  6. bupthcp commented on Feb 26, 2026

    @bupthcp

    Still got the problem on opencode version v1.2.15. Tested it both on Mac and ubuntu 22.04.

  7. giricahunik commented on Mar 8, 2026

    @giricahunik
    Image

    I encountered a similar issue. I am using qwen3.5 9b. It can describe image and put the description into the session/chat title, but its response said the model can not receive image input. I'm confused lol.

  8. mtomas7 commented on Mar 10, 2026

    @mtomas7

    Please see if this config tip will help: #15728 (comment)

  9. humarobot commented on Mar 12, 2026

    @humarobot

    @mtomas7 Excellent!This works!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions