Skip to content

tool/read: full base64 data URI persisted inside tool-result message record (context bloat) #49320

Closed

Description

@MD5Visual

Summary

The v2 read tool persists the entire base64 data URI of an image inside the tool-result record that gets stored in the session's message table — both as the type: "file" content part's uri field and again in the tool's output. Because message records are serialized back into the conversation (and clients render file parts as [file: data:image/jpeg;base64,...]), a single screenshot can inject ~100KB+ of base64 text into context — roughly the image size again — so image-heavy sessions blow past the context limit at ~2x the cost. v1 kept the binary in a separate attachments side-channel and only put a tiny "Image read successfully" string in the output.

Environment

  • opencode version: v2.0.3 (verified against v2.0.4 source at v2.0.4 tag)
  • OS: Darwin 25.6.0 (arm64)
  • Terminal: Apple_Terminal, xterm-256color, truecolor
  • Shell: /bin/zsh
  • Install/channel: latest
  • Active plugins: opencode-dynamic-context-pruning (local plugin), @dietrichgebert/ponytail, opencode-models-discovery@latest

Reproduction

  1. In a v2 session, run the read tool on an image file (e.g., a ~300KB JPEG screenshot).
  2. Inspect the persisted session message for the tool result: SQLite table session_message (column data, look at content[].state.content).
  3. Observe { "type": "file", "uri": "data:image/jpeg;base64,<~400k chars>", "mime": ..., "name": ... } — the complete data URI is stored verbatim, and the tool's output field wraps the same full FileContent again.

Expected Behavior

Binary payloads should not be persisted inside the tool-result text record. The model should still receive the image as real media (image_url parts), but message records should keep a small reference — as v1 did with attachments — so persisted history stays small and clients can render a preview/thumbnail instead of a giant data URI.

Actual Behavior

The full base64 data URI is embedded in the stored message record and rendered verbatim back into agent context. Context/usage accounting then counts it again (model-request.ts boundImages() adds Buffer.byteLength(item.uri) for tool-result file image parts). Net effect: each image effectively costs ~2x in context/usage than the pixels sent to the model.

Additional Context

  • DB evidence from a real session: one assistant message was 127KB; content[1].state.content[1].uri alone was 126,067 chars — the entire data:image/jpeg;base64,... string stored verbatim.
  • v1 (v1.4.14, packages/opencode/src/tool/read.ts) returns { output: "Image read successfully", attachments: [{ type: "file", mime, url }] } — binary in a side-channel, never in the output text.
  • v2 (packages/core/src/tool/plugin/read.ts) builds toModelContent with a file content part whose uri is the full data URI, and also returns that whole FileContent as the tool output.
  • Provider adapters (packages/ai/src/protocols/openai-chat.ts lowerToolMessages/lowerMedia, also openai-responses/anthropic) do correctly lower file parts to image_url media parts — so the model request is fine; this is storage + transcript-rendering bloat, not a provider issue.
  • Suggested fix areas: (1) keep binary payloads out of persisted tool-result records (reuse the v1 attachments pattern / store a file reference); (2) clients should render file parts without inlining data URIs into the transcript text; (3) revisit counting raw URI bytes in boundImages().
  • Frequency: every image read; consistently reproducible.

Activity

  1. github-actions commented on Sep 16, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

  2. MD5Visual commented on Sep 22, 2026

    @MD5Visual
    Author

    Closing as a duplicate of #42263 (base64 media persisted verbatim in tool-result records — the memory/storage angle) and #36343 (image base64 stored twice in the tool-success event — the image-specific duplication).

    Verified against v2.0.14: the read tool still persists the full data:<mime>;base64,... URI in session_message tool-result records (largest in this user's DB: 1,259,239 chars for one PNG). The 20MiB MAX_MEDIA_INGEST_BYTES ingest cap that landed bounds peak memory for oversized files but does not stop image bloat below that size. The architectural fix is tracked in #46173 (managed per-session attachment store, currently open).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions