Repository navigation
Text-only models still receive image read attachments in prompt context #17205
Description
Summary
When a text-only model such as zai-coding-plan/glm-4.7 uses the Read tool on an image, the tool still emits a data:image/...;base64,... attachment. That attachment can move through the message-building path before the provider layer finally rejects image input.
Problem
This is the wrong place to catch the error. By the time the provider rejects the image, the image payload has already entered prompt construction and can cause unnecessary context growth or overflow.
Expected behavior
If the current model does not support image or PDF input, the file should not be attached to the conversation at all. The tool or message layer should emit a short text explanation instead.
Notes
A complete fix should guard both the Read tool path and the message conversion path for tool-result media so unsupported media cannot be reintroduced later by another tool.
Activity
github-actions commentedon Mar 12, 2026
This issue might be a duplicate of existing issues. Please check:
AI_UnsupportedFunctionalityError: 'file part media type application/pdf'when using@ai-sdk/openai-compatible#16338: Similar issue where unsupported file types (PDF) cause errors at the provider layer rather than being caught earliertobwen commentedon Mar 13, 2026
Can this behaviour be enabled/disabled in the config? I wouldn't want huge images to be sent to the LLM (input tokens).
Harryoung commentedon Mar 16, 2026
Agree. Even for models that support multimodal input, it’s not always desirable to push the entire image or PDF into the input context. Large files can easily bloat the prompt and consume tokens without necessarily adding useful signal.
roadmap: #220 filed — Image/vision input is structurally impossible a…
roadmap: #220 filed — Image/vision input is structurally impossible a…
roadmap: ultraworkers#220 filed — Image/vision input is structurally …
roadmap: #220 filed — Image/vision input is structurally impossible a…
roadmap: ultraworkers#220 filed — Image/vision input is structurally …
roadmap: #220 filed — Image/vision input is structurally impossible a…
roadmap: #220 filed — Image/vision input is structurally impossible a…
roadmap: #220 filed — Image/vision input is structurally impossible a…
github-actions commentedon Jul 3, 2026
To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.