Feature Request: Automatic Image Vision Processing for Non-Vision Models

Problem

When users paste images directly into the OpenCode input field, OpenCode sends the image data to the AI model. However, many models (like MiniMax via API) don't support image input, resulting in errors like:

ERROR: Cannot read "image.png" (this model does not support image input)

This creates a poor user experience where:

User pastes an image
OpenCode sends it to the model
Model rejects it with an error
User has no way to analyze the image

Proposed Solution

Add an automatic image processing feature that:

Detects when a user pastes an image (vs. text)
Checks if the target model supports vision (via model capabilities config)
If model doesn't support vision:
- Automatically call a local vision API (e.g., Ollama with vision model, or local Qwen2.5-Vision)
- Get the image description
- Send the description as text to the target model

Alternative: Vision API Integration

Allow users to configure a fallback vision provider in their OpenCode config:

{
  "vision": {
    "provider": "ollama",  // or "openai", "anthropic", etc.
    "model": "qwen2.5-vision",
    "endpoint": "http://localhost:11434"
  },
  "visionFallback": true  // auto-use vision provider for image analysis
}

Benefits

Better UX - Images just work, even with non-vision models
Privacy - Keep image processing local (using Ollama)
Flexibility - Users can choose their preferred vision provider
Backward compatible - Only activates when vision fallback is enabled

Implementation Hints

Hook into the message composition/pre-send phase
Check userMessage.content for image data (base64 or file reference)
If settings.visionFallback === true, intercept and process images
Call configured vision API and replace image content with description

Environment

OpenCode version: 1.1.61
OS: macOS
Model: MiniMax (via API, no vision support)

This issue doesn't fully meet our contributing guidelines.

What needs to be fixed:

Feature request titles must start with [FEATURE]: (current title uses "Feature Request:" instead)
Feature requests require the verification checkbox confirming the feature hasn't been suggested before

Please edit this issue to address the above within 2 hours, or it will be automatically closed.

This issue might be a duplicate of existing issues. Please check:

Clipboard images routed directly to model instead of MCP when model lacks vision support #18437: Clipboard images routed directly to model instead of MCP when model lacks vision support — describes the exact same error (Cannot read "image.png" (this model does not support image input)) and requests a fallback routing mechanism for non-vision models
[FEATURE]:Pasted images use data: URL instead of file path, breaking MCP tools for non-vision models #14673: Pasted images use data: URL instead of file path, breaking MCP tools for non-vision models — related request for image handling when the model lacks vision support
[FEATURE]:Support for direct copy and paste of images. The current situation is that direct copying will go through the large model and will not trigger mcp. #19853: Support for direct copy and paste of images with MCP fallback — similar request for automatic image routing

If you believe this was flagged incorrectly, please let a maintainer know.

This isn't a new issue, although I do really want to move forward with solving it. Mimo v2 Pro (which doesn't support image recognition) should be able to invoke Mimo Omni when it detects an image.

Actually, Minimax has an MCP for image recognition that follows exactly this approach, but it seems that due to Opencode limitations, the image path must be explicitly specified for it to call the corresponding tool. Also, does Opencode currently have some automatic image translation logic? When I try to paste a path, it pastes the file itself.

This issue has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window.

Feel free to open a new issue that follows our issue templates.

Navigation Menu

Repository navigation

Feature Request: Automatic Image Vision Processing for Non-Vision Models #21004