Skip to content

feat: vision sub-model fallback for non-multimodal models #31936

Closed as not planned

Description

@Tan-zixuan

Summary

When the primary model doesn't support image input (e.g., mimo-v2.5-pro has image: false in models.dev), the TUI blocks clipboard paste and drag-drop with "this model does not support image input". There is no way to configure a fallback vision model.

Motivation

Many strong coding models (like mimo-v2.5-pro) don't support images natively, but their providers often have a companion vision model (like mimo-v2.5 or mimo-v2-omni). Currently users must choose between:

  • A strong non-vision model (no image support)
  • A weaker vision-capable model

This is a common pattern in the ecosystem — both Hermes and Claude Code solve this with a dedicated vision sub-model configuration:

  • Hermes: auxiliary.vision config section with separate provider, model, base_url, api_key
  • Claude Code: automatic vision sub-agent delegation

Proposed Solution

Add a vision_model or auxiliary.vision configuration option that specifies a fallback model for image processing:

{
  "model": "xiaomi/mimo-v2.5-pro",  // main model (no image support)
  "vision_model": "xiaomi/mimo-v2.5"  // fallback for images
}

When the user pastes an image or drags in a file:

  1. If the main model supports images → send directly (current behavior)
  2. If not, and vision_model is configured → send image to vision model, get text description, pass to main model
  3. If neither → show current error

Alternatives Considered

  • Switching to a vision-capable model entirely (loses reasoning quality)
  • Using a custom tool/script to pre-process images (poor UX)

Additional Context

This affects any provider where the strongest model lacks multimodal support but a weaker model has it. Common with Xiaomi/MiMo models, and likely other providers too.

Activity

  1. github-actions commented on Jun 11, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    Please review those issues (especially #24948, which even references a linked PR) and consider adding your input there rather than opening a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions