[FEATURE]: Auto image-to-text transcription via multimodal model for non-multimodal providers #22828

• I have searched for existing issues and confirmed this is not a duplicate
• I have read the contributing guidelines

Problem

When using a non-multimodal LLM provider (e.g., text-only models) in OpenCode, sending an image attachment results in the model being unable to understand or process the image. There is no automatic fallback mechanism to:

1. Detect that the current model does not support image input
2. Automatically route the image to a configured multimodal model for transcription
3. Convert the image content to text description
4. Pass the text description to the primary non-multimodal model

Current Behavior

• User sends an image in the conversation
• If the active model does not support image/vision input, the image is either ignored or causes an error
• No automatic image transcription fallback exists

Expected Behavior

When a non-multimodal model receives an image, OpenCode should:

1. Detect the model's modalities do not include image input
2. Use a separately configured image caption model (multimodal model) to describe the image
3. Replace the image with the text description in the context
4. Continue the conversation with the primary model using the transcribed text

Suggested Configuration

Add a new top-level config option in opencode.json:

{
  "image_transcription_model": "dmxapi/glm-4.1v-thinking-flash"
}

Or within the provider's model definition, allow marking a model as the image transcription provider:

{
  "provider": {
    "dmxapi": {
      "models": {
        "glm-5.1": {
          "name": "glm-5.1"
        },
        "glm-4.1v-thinking-flash": {
          "name": "glm-4.1v-thinking-flash",
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          },
          "use_for_image_transcription": true
        }
      }
    }
  }
}

Related Issues

• Feature Request: Automatic Image Vision Processing for Non-Vision Models #21004: Similar request but was closed as out of scope. However, this feature is increasingly important as more users combine cost-effective text-only models with occasional image understanding needs.
• Clipboard images routed directly to model instead of MCP when model lacks vision support #18437: Related issue about image routing when model lacks vision support.

Unlike those previous requests, this proposal includes a concrete configuration design and leverages the existing modalities field already in the schema, making it straightforward to implement.

Use Case

Many users use cost-effective text-only models as their primary coding assistant, but still need image understanding (e.g., reading screenshots, diagrams, UI mockups). Having to manually switch models for image-related tasks is inconvenient. An automatic image transcription pipeline would provide the best of both worlds.

Environment

• OpenCode version: latest (via npm)
• OS: Windows
• Primary model: text-only (via OpenAI-compatible provider)
• Available multimodal model: GLM-4.1V-Thinking-Flash (via same provider)