ai: OpenAI rejects PDF in tool-result replay #44486

Summary

Reading a valid small PDF with the read tool can permanently fail an OpenAI Responses session on the next turn. OpenCode replays the PDF inside function_call_output.output as input_file.file_data, and OpenAI rejects it with HTTP 400.

Environment

• opencode version: opencode2 v0.0.0-beta-17963
• OS: Darwin 25.5.0 arm64
• Terminal: TERM_PROGRAM=ghostty, TERM=xterm-256color, COLORTERM=truecolor
• Shell: /bin/zsh
• Install/channel: beta
• Active plugins: opencode-gpt-imagegen (not involved in the PDF read or request conversion)

Reproduction

1. Start a session using native OpenAI Responses with an attachment-capable OpenAI model (openai/gpt-5.6-sol-fast in the observed failure).
2. Ask the agent to read a valid local PDF using the built-in read tool. The observed files were PDF 1.4 documents with two pages and base64 payloads of approximately 23 KB.
3. Allow the agent to continue after the tool returns PDF read successfully plus the data:application/pdf;base64,... file content.
4. Observe that the next model request fails, and subsequent retries replaying the same history fail identically.

Expected Behavior

The PDF should either be accepted by OpenAI or moved out of the function tool result into a provider-supported user file message so the session can continue.

Actual Behavior

The next OpenAI Responses request fails with HTTP 400:

Invalid 'input[209].output[1].file_data'.

The server log records:

Failed to drain Session
AI.Error: RequestExecutor.execute: Invalid 'input[209].output[1].file_data'.

The failure is deterministic when retrying the persisted session. Earlier retries reported the same error at input[195].output[1].file_data; reverting some history only changed the input index.

Additional Context

The stored tool result contained three valid PDF data URLs. Each payload decoded successfully and was identified as a PDF 1.4, two-page document. This is not the large-base64 validation issue fixed in #42921: these payloads were small, passed local conversion, reached OpenAI, and were rejected remotely.

Current dev keeps PDF attachments inside tool results for @ai-sdk/openai:

if (model.api.npm === "@ai-sdk/openai") return true

That path eventually becomes an AI SDK Responses function_call_output.output item:

{
  "type": "input_file",
  "filename": "document.pdf",
  "file_data": "data:application/pdf;base64,..."
}

There is an unmerged fallback implementation on origin/aisdk-media-fallback (2e0f414e78, fix(core): relocate AI SDK tool media) that moves images and PDFs from tool results into a synthetic user message. That approach avoids putting PDF input_file content under function_call_output.output.

Related but distinct reports:

• ai: OpenAI Responses rejects valid large PDF base64 during tool-result replay #42914 failed locally on a large valid base64 payload before provider submission.
• bug: tool-result media 400s whole request for models without attachment capability #41163 concerns tool-result media sent to models whose attachment capability is disabled.