Config option to keep generated-image base64 out of the conversation history (custom providers hit 1009/413 on full-history resends) #37194

What variant of Codex are you using?

CLI (custom model provider, wire_api = "responses")

What feature would you like to see?

An opt-in config to keep generated-image base64 out of the durable conversation history, since the image is already saved to disk:

[tools.image_generation]
embed_results_in_history = false  # default: true (current behavior)

Problem. The imagegen extension already persists every generated image to <save_root>/generated_images/<thread>/<call>.png and appends a saved-path hint to the tool output. However, GeneratedImageOutput::to_response_item() also unconditionally embeds the full base64 result as an input_image content item into the durable history (tool.rs), and there is no way to turn that off.

For custom model providers this compounds badly, because they run with store: false and resend the full transcript:

• every HTTP turn resends every previously generated image (a single gpt-image-2 PNG is typically 2–8 MB, ×1.33 as base64);
• with supports_websockets = true, incremental sends help while a socket lives, but the first request after any reconnect (including the upstream 60-minute connection recycle) or after a compaction/history rebuild resends the full transcript in one frame;
• once the transcript holds a few images, those full-history requests exceed common gateway/transport limits. Observed failure chain in a real session (~35 MiB of image data URLs): websocket close 1009 → retries exhausted → HTTP fallback → 413 Payload Too Large. The session can only be recovered by starting a new one.

Token-wise the embedded copy is also re-billed as image input tokens on every full-history resend, even though the model rarely needs to re-see old generations: follow-up edits go through referenced_image_paths, which reads the saved file from disk and attaches it to the Images API request directly.

Proposed behavior (working implementation linked below):

• embed_results_in_history = true (default): exactly today's behavior.
• embed_results_in_history = false: when the artifact was saved successfully, record only a note in the tool output — saved path + guidance to use referenced_image_paths for edits + the existing "already displayed to the user" caveat. When the artifact save failed (no save_root, I/O error), embed the image regardless of the flag so the model never loses its only copy.
• UI display is unaffected: the ImageGenerationEnd event still carries the result bytes.

Known trade-offs, acceptable behind an explicit opt-in:

• the model cannot re-inspect earlier generated images unless it re-reads the saved file;
• num_last_images_to_include will not find omitted generations in history (the note steers the model to referenced_image_paths instead).

Additional information

I have a tested implementation on a fork, happy to submit it as a PR if invited per the contribution guidelines: sokdak/codex@main...sokdak:codex:feat/imagegen-embed-results-toggle

It follows the existing [tools.*] toggle conventions (UpdatePlanToolConfig pattern), threads the flag through ImageGenerationExtensionConfig, branches only in to_response_item(), and includes unit tests plus the regenerated config.schema.json. cargo test -p codex-image-generation-extension (12 passed) and cargo test -p codex-core --lib config:: (474 passed, including the schema fixture) are green.

Context: this was root-caused while operating a pooled Codex gateway; the full analysis is in https://github.com/sokdak/codex-oauth-pool/issues/52.