Local compaction v2 retains unbounded input_image payloads, causing repeated auto-compaction #33493

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   appIssues related to the Codex desktop appIssues related to the Codex desktop app

   contextIssues related to context management (including compaction)Issues related to context management (including compaction)

   on Jul 16, 2026
2. github-actions commented on Jul 16, 2026

   

   github-actionsbot

   on Jul 16, 2026 – with GitHub Actions

   Contributor

   More actions

   Potential duplicates detected. Please review them and close your issue if it is a duplicate.

   • Image-heavy multi-agent task replays inherited image context, causing 1.48B tokens, 70.73GB traffic, and heavy swap/disk growth #33235

   Powered by Codex Action
3. ignatremizov commented on Jul 19, 2026

   

   ignatremizov

   on Jul 19, 2026

   Last edited by ignatremizov

   More actions

   Additional current reproduction and source trace (2026-07-19).

   Reproduction

   I inspected another long-lived image-heavy thread after a local compaction-v2 checkpoint. Sanitized structural measurements from the latest compacted.payload.replacement_history:

   Metric	Observed value
   Rollout JSONL size	2,873,784,216 bytes
   Compaction records	54
   Replacement-history items	211
   User messages	209
   Retained input_image parts	142
   Retained image detail	142 × high
   Retained inline image/data-URL characters	111,563,732
   Retained text characters	56,733
   Effective model context window	353,400 tokens (gpt-5.6-sol)

   The token sequence makes the impact visible:

   1. Immediately after compaction, local recomputation emitted total_tokens=294,141, input_tokens=0, which rendered as 17% context remaining.
   2. The first real post-compaction model response reported input_tokens=342,406, output_tokens=230, total_tokens=342,636, which rendered as 3% remaining.

   The apparent drop was not caused by the file-read output displayed immediately before the second token event. That usage belongs to the request that generated the read call, so the result could not yet have been part of its input. The first server-provided usage corrected the lower-bound local estimate for the already-installed image-heavy prompt.

   No raw rollout is attached because it contains private prompts, paths, tool output, and images. The counts above were computed structurally without printing content.

   All 142 retained images in this reproduction were already surrounded by the canonical local-image text wrapper: an opening <image ... path="..."> item immediately before the InputImage and a closing </image> immediately after it. The compactor can therefore remove the inline bytes without losing the model-callable source path.

   Current upstream source analysis

   Current upstream main now uses a 64k retained-message budget, but the budget remains text-only:

   codex/codex-rs/core/src/compact_remote_v2.rs

   Lines 52 to 54 in 0fb559f

   	// Mirror the current /responses/compact retained-message default while the
   	// server-side path remains the reference implementation.
   	const RETAINED_MESSAGE_TOKEN_BUDGET: usize = 64_000;

   build_v2_compacted_history selects retained user/developer/system messages from the compaction prompt and passes them through the retained-message truncator:

   codex/codex-rs/core/src/compact_remote_v2.rs

   Lines 443 to 460 in 0fb559f

   	fn build_v2_compacted_history(
   	prompt_input: &[ResponseItem],
   	compaction_output: ResponseItem,
   	) -> (Vec<ResponseItem>, usize) {
   	let retained = prompt_input
   	.iter()
   	.filter(|item| is_retained_for_remote_compaction_v2(item))
   	.filter(|item| should_keep_compacted_history_item(item))
   	.cloned()
   	.collect::<Vec<_>>();
   	let mut retained =
   	truncate_retained_messages_for_remote_compaction(retained, RETAINED_MESSAGE_TOKEN_BUDGET);
   	let retained_image_count = retained
   	.iter()
   	.map(retained_input_image_count)
   	.sum::<usize>();
   	retained.push(compaction_output);
   	(retained, retained_image_count)

   The truncator charges only text. InputImage and InputAudio contribute zero, and the partial-message path always preserves them:

   codex/codex-rs/core/src/compact_remote_v2.rs

   Lines 482 to 560 in 0fb559f

   	fn truncate_retained_messages_for_remote_compaction(
   	items: Vec<ResponseItem>,
   	max_tokens: usize,
   	) -> Vec<ResponseItem> {
   	let mut remaining = max_tokens;
   	let mut truncated_reversed = Vec::with_capacity(items.len());
   	for item in items.into_iter().rev() {
   	if remaining == 0 {
   	continue;
   	}
   	let token_count = message_text_token_count(&item).max(1);
   	if token_count <= remaining {
   	truncated_reversed.push(item);
   	remaining = remaining.saturating_sub(token_count);
   	} else if let Some(truncated_item) =
   	truncate_message_text_to_token_budget(item, /*max_tokens*/ remaining)
   	{
   	truncated_reversed.push(truncated_item);
   	remaining = 0;
   	}
   	}
   	truncated_reversed.reverse();
   	truncated_reversed
   	}
   	fn message_text_token_count(item: &ResponseItem) -> usize {
   	let ResponseItem::Message { content, .. } = item else {
   	return 0;
   	};
   	content
   	.iter()
   	.map(|item| match item {
   	ContentItem::InputText { text } | ContentItem::OutputText { text } => {
   	approx_token_count(text)
   	}
   	ContentItem::InputImage { .. } | ContentItem::InputAudio { .. } => 0,
   	})
   	.sum()
   	}
   	fn truncate_message_text_to_token_budget(
   	item: ResponseItem,
   	max_tokens: usize,
   	) -> Option<ResponseItem> {
   	let ResponseItem::Message {
   	id,
   	role,
   	content,
   	phase,
   	internal_chat_message_metadata_passthrough: metadata,
   	} = item
   	else {
   	return Some(item);
   	};
   	let mut remaining = max_tokens;
   	let mut truncated_content = Vec::with_capacity(content.len());
   	for mut content_item in content {
   	match &mut content_item {
   	ContentItem::InputText { text } | ContentItem::OutputText { text } => {
   	if remaining == 0 {
   	continue;
   	}
   	let token_count = approx_token_count(text);
   	if token_count <= remaining {
   	remaining = remaining.saturating_sub(token_count);
   	} else {
   	*text = truncate_text(text, TruncationPolicy::Tokens(remaining));
   	remaining = 0;
   	}
   	if !text.is_empty() {
   	truncated_content.push(content_item);
   	}
   	}
   	ContentItem::InputImage { .. } | ContentItem::InputAudio { .. } => {
   	truncated_content.push(content_item);

   The current tests explicitly lock in this behavior by asserting that images survive truncation and that an image-only message costs only the .max(1) message floor:

   codex/codex-rs/core/src/compact_remote_v2.rs

   Lines 729 to 819 in 0fb559f

   	#[test]
   	fn retained_history_truncation_preserves_images_and_truncates_later_text_parts() {
   	let item = ResponseItem::Message {
   	id: None,
   	role: "user".to_string(),
   	content: vec![
   	ContentItem::InputText {
   	text: "abcdef".to_string(),
   	},
   	ContentItem::InputImage {
   	image_url: "data:image/png;base64,abc".to_string(),
   	detail: None,
   	},
   	ContentItem::OutputText {
   	text: "uvwxyz".to_string(),
   	},
   	],
   	phase: None,
   	internal_chat_message_metadata_passthrough: None,
   	};
   	let truncated =
   	truncate_retained_messages_for_remote_compaction(vec![item], /*max_tokens*/ 3);
   	assert_eq!(
   	truncated,
   	vec![ResponseItem::Message {
   	id: None,
   	role: "user".to_string(),
   	content: vec![
   	ContentItem::InputText {
   	text: "abcdef".to_string(),
   	},
   	ContentItem::InputImage {
   	image_url: "data:image/png;base64,abc".to_string(),
   	detail: None,
   	},
   	ContentItem::OutputText {
   	text: "uv…1 tokens truncated…yz".to_string(),
   	},
   	],
   	phase: None,
   	internal_chat_message_metadata_passthrough: None,
   	}]
   	);
   	}
   	#[test]
   	fn retained_history_truncation_charges_image_only_messages() {
   	let image_only_message = ResponseItem::Message {
   	id: None,
   	role: "user".to_string(),
   	content: vec![ContentItem::InputImage {
   	image_url: "data:image/png;base64,abc".to_string(),
   	detail: None,
   	}],
   	phase: None,
   	internal_chat_message_metadata_passthrough: None,
   	};
   	let newest = message("user", "new", /*phase*/ None);
   	let retained = vec![
   	message("user", "old", /*phase*/ None),
   	image_only_message.clone(),
   	newest.clone(),
   	];
   	let truncated =
   	truncate_retained_messages_for_remote_compaction(retained, /*max_tokens*/ 2);
   	assert_eq!(truncated, vec![image_only_message, newest]);
   	}
   	#[test]
   	fn retained_history_truncation_drops_image_only_messages_after_budget_is_spent() {
   	let image_only_message = ResponseItem::Message {
   	id: None,
   	role: "user".to_string(),
   	content: vec![ContentItem::InputImage {
   	image_url: "data:image/png;base64,abc".to_string(),
   	detail: None,
   	}],
   	phase: None,
   	internal_chat_message_metadata_passthrough: None,
   	};
   	let newest = message("user", "new", /*phase*/ None);
   	let retained = vec![image_only_message, newest.clone()];
   	let truncated =
   	truncate_retained_messages_for_remote_compaction(retained, /*max_tokens*/ 1);
   	assert_eq!(truncated, vec![newest]);

   Local-image prompt construction already emits the source path in an adjacent model-visible wrapper and keeps the raw image as a separate content item:

   codex/codex-rs/protocol/src/models.rs

   Lines 1392 to 1400 in 0fb559f

   	pub fn local_image_open_tag_text_with_path(label_number: usize, path: &std::path::Path) -> String {
   	let label = local_image_label_text(label_number);
   	let path = path.display();
   	format!("{LOCAL_IMAGE_OPEN_TAG_PREFIX}{label} path=\"{path}\"{LOCAL_IMAGE_OPEN_TAG_SUFFIX}")
   	}
   	pub fn is_local_image_open_tag_text(text: &str) -> bool {
   	text.strip_prefix(LOCAL_IMAGE_OPEN_TAG_PREFIX)
   	.is_some_and(|rest| rest.ends_with(LOCAL_IMAGE_OPEN_TAG_SUFFIX))

   codex/codex-rs/protocol/src/models.rs

   Lines 1568 to 1588 in 0fb559f

   	fn local_image_content_items(
   	path: &std::path::Path,
   	image_url: String,
   	label_number: Option<usize>,
   	detail: ImageDetail,
   	) -> Vec<ContentItem> {
   	let mut items = Vec::with_capacity(3);
   	if let Some(label_number) = label_number {
   	items.push(ContentItem::InputText {
   	text: local_image_open_tag_text_with_path(label_number, path),
   	});
   	}
   	items.push(ContentItem::InputImage {
   	image_url,
   	detail: Some(detail),
   	});
   	if label_number.is_some() {
   	items.push(ContentItem::InputText {
   	text: LOCAL_IMAGE_CLOSE_TAG.to_string(),
   	});
   	}

   view_image already accepts a model-generated path plus an optional environment ID and resolves it through the selected environment filesystem:

   codex/codex-rs/core/src/tools/handlers/view_image.rs

   Lines 53 to 57 in 0fb559f

   	struct ViewImageArgs {
   	path: String,
   	#[serde(default)]
   	environment_id: Option<String>,
   	detail: Option<String>,

   codex/codex-rs/core/src/tools/handlers/view_image.rs

   Lines 118 to 168 in 0fb559f

   	let ViewImageArgs {
   	path,
   	environment_id,
   	detail,
   	} = parse_arguments(&arguments)?;
   	// `high` is the explicit spelling of the default resized path.
   	// Other string values remain invalid rather than being silently reinterpreted.
   	let detail = match detail.as_deref() {
   	None => None,
   	Some("high") => Some(ViewImageDetail::High),
   	Some("original") => Some(ViewImageDetail::Original),
   	Some(detail) => {
   	return Err(FunctionCallError::RespondToModel(format!(
   	"view_image.detail only supports `high` or `original`; omit `detail` for default high resized behavior, got `{detail}`"
   	)));
   	}
   	};
   	let Some(turn_environment) =
   	resolve_tool_environment(&step_context.environments, environment_id.as_deref())?
   	else {
   	return Err(FunctionCallError::RespondToModel(
   	"view_image is unavailable in this session".to_string(),
   	));
   	};
   	let path_uri = turn_environment.cwd().join(&path).map_err(|err| {
   	FunctionCallError::RespondToModel(format!(
   	"unable to resolve image path `{path}` against environment cwd `{}`: {err}",
   	turn_environment.cwd(),
   	))
   	})?;
   	let model_visible_path = path_uri.inferred_native_path_string();
   	let sandbox = turn
   	.file_system_sandbox_context(/*additional_permissions*/ None, turn_environment);
   	let fs = turn_environment.environment.get_filesystem();
   	let metadata = fs
   	.get_metadata(&path_uri, Some(&sandbox))
   	.await
   	.map_err(|error| {
   	FunctionCallError::RespondToModel(format!(
   	"unable to locate image at `{model_visible_path}`: {error}"
   	))
   	})?;
   	if !metadata.is_file {
   	return Err(FunctionCallError::RespondToModel(format!(
   	"image path `{model_visible_path}` is not a file"
   	)));
   	}
   	let file_bytes = fs

   There is already a content-aware image estimator elsewhere in core. It replaces inline base64 payload size with an estimated model-visible image cost (approximately 1,844 tokens for ordinary resized images, with separate original-detail patch accounting):

   codex/codex-rs/core/src/context_manager/history.rs

   Lines 521 to 579 in 0fb559f

   	fn estimate_item_token_count(item: &ResponseItem) -> i64 {
   	let model_visible_bytes = estimate_response_item_model_visible_bytes(item);
   	approx_tokens_from_byte_count_i64(model_visible_bytes)
   	}
   	/// Approximate model-visible byte cost for one image input.
   	///
   	/// The estimator later converts bytes to tokens using a 4-bytes/token heuristic
   	/// with ceiling division, so 7,373 bytes maps to approximately 1,844 tokens.
   	const RESIZED_IMAGE_BYTES_ESTIMATE: i64 = 7373;
   	// See https://platform.openai.com/docs/guides/images-vision#calculating-costs.
   	// Use a direct 32px patch count only for `detail: "original"`;
   	// all other image inputs continue to use `RESIZED_IMAGE_BYTES_ESTIMATE`.
   	const ORIGINAL_IMAGE_PATCH_SIZE: u32 = 32;
   	// See https://platform.openai.com/docs/guides/images-vision#model-sizing-behavior.
   	// Keep this hard-coded for now; move it into model capabilities if the patch
   	// budget starts changing often across model releases.
   	const ORIGINAL_IMAGE_MAX_PATCHES: usize = 10_000;
   	const ORIGINAL_IMAGE_ESTIMATE_CACHE_SIZE: usize = 32;
   	static ORIGINAL_IMAGE_ESTIMATE_CACHE: LazyLock<BlockingLruCache<[u8; 20], Option<i64>>> =
   	LazyLock::new(|| {
   	BlockingLruCache::new(
   	NonZeroUsize::new(ORIGINAL_IMAGE_ESTIMATE_CACHE_SIZE).unwrap_or(NonZeroUsize::MIN),
   	)
   	});
   	fn estimate_response_item_model_visible_bytes(item: &ResponseItem) -> i64 {
   	match item {
   	ResponseItem::Reasoning {
   	encrypted_content: Some(content),
   	..
   	}
   	| ResponseItem::Compaction {
   	encrypted_content: content,
   	..
   	}
   	| ResponseItem::ContextCompaction {
   	encrypted_content: Some(content),
   	..
   	} => i64::try_from(estimate_reasoning_length(content.len())).unwrap_or(i64::MAX),
   	item => {
   	let raw = serde_json::to_string(item)
   	.map(|serialized| i64::try_from(serialized.len()).unwrap_or(i64::MAX))
   	.unwrap_or_default();
   	let (image_payload_bytes, image_replacement_bytes) =
   	image_data_url_estimate_adjustment(item);
   	let (encrypted_payload_bytes, encrypted_replacement_bytes) =
   	encrypted_function_output_estimate_adjustment(item);
   	// Replace raw base64 payload bytes with a per-image estimate.
   	// We intentionally preserve the data URL prefix and JSON
   	// wrapper bytes already included in `raw`.
   	let raw = raw
   	.saturating_sub(image_payload_bytes)
   	.saturating_add(image_replacement_bytes);
   	raw.saturating_sub(encrypted_payload_bytes)
   	.saturating_add(encrypted_replacement_bytes)
   	}
   	}

   codex/codex-rs/core/src/context_manager/history.rs

   Lines 652 to 679 in 0fb559f

   	/// Scans one response item for discount-eligible inline image data URLs and
   	/// returns:
   	/// - total base64 payload bytes to subtract from raw serialized size
   	/// - total replacement byte estimate for those images
   	fn image_data_url_estimate_adjustment(item: &ResponseItem) -> (i64, i64) {
   	let mut payload_bytes = 0i64;
   	let mut replacement_bytes = 0i64;
   	let mut accumulate = |image_url: &str, detail: Option<ImageDetail>| {
   	if let Some(payload_len) = parse_base64_image_data_url(image_url).map(str::len) {
   	payload_bytes =
   	payload_bytes.saturating_add(i64::try_from(payload_len).unwrap_or(i64::MAX));
   	replacement_bytes = replacement_bytes.saturating_add(match detail {
   	Some(ImageDetail::Original) => {
   	estimate_original_image_bytes(image_url).unwrap_or(RESIZED_IMAGE_BYTES_ESTIMATE)
   	}
   	_ => RESIZED_IMAGE_BYTES_ESTIMATE,
   	});
   	}
   	};
   	match item {
   	ResponseItem::Message { content, .. } => {
   	for content_item in content {
   	if let ContentItem::InputImage { image_url, detail } = content_item {
   	accumulate(image_url, *detail);
   	}
   	}

   That estimator is used after history replacement for the provisional token-count event, but not for retained-message selection. recompute_token_usage also estimates only base instructions plus installed history and marks the result as input_tokens=0, explaining why the first real server count can jump materially:

   codex/codex-rs/core/src/session/mod.rs

   Lines 3668 to 3704 in 0fb559f

   	pub(crate) async fn recompute_token_usage(&self, turn_context: &TurnContext) {
   	let history = self.clone_history().await;
   	let base_instructions = self.get_base_instructions().await;
   	let Some(estimated_total_tokens) =
   	history.estimate_token_count_with_base_instructions(&base_instructions)
   	else {
   	return;
   	};
   	{
   	let mut state = self.state.lock().await;
   	let mut info = state.token_info().unwrap_or(TokenUsageInfo {
   	total_token_usage: TokenUsage::default(),
   	last_token_usage: TokenUsage::default(),
   	model_context_window: None,
   	});
   	info.last_token_usage = TokenUsage {
   	input_tokens: 0,
   	cached_input_tokens: 0,
   	cache_write_input_tokens: 0,
   	output_tokens: 0,
   	reasoning_output_tokens: 0,
   	total_tokens: estimated_total_tokens.max(0),
   	};
   	if let Some(model_context_window) = turn_context.model_context_window() {
   	info.model_context_window = Some(model_context_window);
   	}
   	state.set_token_info(Some(info));
   	}
   	self.set_auto_compact_window_estimated_prefill_for_scope(
   	turn_context,
   	estimated_total_tokens,
   	)
   	.await;
   	self.send_token_count_event(turn_context).await;

   Historic rollouts currently receive no equivalent cleanup. Reconstruction selects the latest surviving replacement_history as its base and installs it exactly before replaying the suffix:

   codex/codex-rs/core/src/session/rollout_reconstruction.rs

   Lines 113 to 185 in 0fb559f

   	pub(super) async fn reconstruct_history_from_rollout(
   	&self,
   	turn_context: &TurnContext,
   	rollout_items: &[RolloutItem],
   	) -> RolloutReconstruction {
   	// Replay metadata should already match the shape of the future lazy reverse loader, even
   	// while history materialization still uses an eager bridge. Scan newest-to-oldest,
   	// stopping once a surviving replacement-history checkpoint and the required resume metadata
   	// are both known; then replay only the buffered surviving tail forward to preserve exact
   	// history semantics.
   	let has_legacy_compaction_without_window_number =
   	rollout_items.iter().any(|item| {
   	matches!(item, RolloutItem::Compacted(compacted) if compacted.window_number.is_none())
   	});
   	let initial_window = if has_legacy_compaction_without_window_number {
   	None
   	} else {
   	rollout_items.iter().find_map(|item| match item {
   	RolloutItem::SessionMeta(session_meta) => session_meta
   	.meta
   	.context_window
   	.as_ref()
   	.and_then(reconstructed_window_from_session_context_window),
   	_ => None,
   	})
   	};
   	let mut base_replacement_history: Option<&[ResponseItem]> = None;
   	let mut previous_turn_settings = None;
   	let mut reference_context_item = TurnReferenceContextItem::NeverSet;
   	let mut world_state_replay = Vec::new();
   	let mut window = None;
   	// Rollback is "drop the newest N user turns". While scanning in reverse, that becomes
   	// "skip the next N user-turn segments we finalize".
   	let mut pending_rollback_turns = 0usize;
   	// Borrowed suffix of rollout items newer than the newest surviving replacement-history
   	// checkpoint. If no such checkpoint exists, this remains the full rollout.
   	let mut rollout_suffix = rollout_items;
   	// Reverse replay accumulates rollout items into the newest in-progress turn segment until
   	// we hit its matching `TurnStarted`, at which point the segment can be finalized.
   	let mut active_segment: Option<ActiveReplaySegment<'_>> = None;
   	for (index, item) in rollout_items.iter().enumerate().rev() {
   	match item {
   	RolloutItem::Compacted(compacted) => {
   	let active_segment =
   	active_segment.get_or_insert_with(ActiveReplaySegment::default);
   	active_segment.world_state_replay.push(item);
   	if active_segment.window.is_none()
   	&& let Some(window_number) = compacted.window_number
   	{
   	active_segment.window = Some(ReconstructedWindow {
   	number: window_number,
   	first_id: compacted.first_window_id.as_deref().and_then(parse_uuid_v7),
   	previous_id: compacted
   	.previous_window_id
   	.as_deref()
   	.and_then(parse_uuid_v7),
   	id: compacted.window_id.as_deref().and_then(parse_uuid_v7),
   	});
   	}
   	// Looking backward, compaction clears any older baseline unless a newer
   	// `TurnContextItem` in this same segment has already re-established it.
   	if matches!(
   	active_segment.reference_context_item,
   	TurnReferenceContextItem::NeverSet
   	) {
   	active_segment.reference_context_item = TurnReferenceContextItem::Cleared;
   	}
   	if active_segment.base_replacement_history.is_none()
   	&& let Some(replacement_history) = &compacted.replacement_history
   	{
   	active_segment.base_replacement_history = Some(replacement_history);
   	rollout_suffix = &rollout_items[index + 1..];

   codex/codex-rs/core/src/session/rollout_reconstruction.rs

   Lines 317 to 345 in 0fb559f

   	let mut history = ContextManager::new();
   	let mut saw_legacy_compaction_without_replacement_history = false;
   	if let Some(base_replacement_history) = base_replacement_history {
   	history.replace(base_replacement_history.to_vec());
   	}
   	// Materialize exact history semantics from the replay-derived suffix. The eventual lazy
   	// design should keep this same replay shape, but drive it from a resumable reverse source
   	// instead of an eagerly loaded `&[RolloutItem]`.
   	for item in rollout_suffix {
   	match item {
   	RolloutItem::ResponseItem(response_item) => {
   	history.record_items(
   	std::iter::once(response_item),
   	turn_context.model_info.truncation_policy.into(),
   	);
   	}
   	RolloutItem::InterAgentCommunication(communication) => {
   	let response_item = communication.to_model_input_item();
   	history.record_items(
   	std::iter::once(&response_item),
   	turn_context.model_info.truncation_policy.into(),
   	);
   	}
   	RolloutItem::InterAgentCommunicationMetadata { .. } => {}
   	RolloutItem::Compacted(compacted) => {
   	if let Some(replacement_history) = &compacted.replacement_history {
   	// This should actually never happen, because the reverse loop above (to build rollout_suffix)
   	// should stop before any compaction that has Some replacement_history
   	history.replace(replacement_history.clone());

   Both resume and fork use the shared reconstruction path. It currently prepares legacy media in memory while explicitly leaving the recorded rollout unchanged, so image-heavy checkpoints remain active and are processed again on every reconstruction:

   codex/codex-rs/core/src/session/mod.rs

   Lines 1248 to 1333 in 0fb559f

   	session_configuration.thread_source.as_ref(),
   	Some(ThreadSource::Subagent)
   	),
   	)
   	};
   	let has_prior_user_turns = initial_history_has_prior_user_turns(&conversation_history);
   	{
   	let mut state = self.state.lock().await;
   	state.set_next_turn_is_first(!has_prior_user_turns);
   	}
   	match conversation_history {
   	InitialHistory::New | InitialHistory::Cleared => {
   	// Defer initial context insertion until the first real turn starts so
   	// turn/start overrides can be merged before we write model-visible context.
   	self.set_previous_turn_settings(/*previous_turn_settings*/ None)
   	.await;
   	}
   	InitialHistory::Resumed(resumed_history) => {
   	let turn_context = self.new_default_turn().await;
   	let rollout_items = resumed_history.history;
   	let previous_turn_settings = self
   	.apply_rollout_reconstruction(&turn_context, &rollout_items)
   	.await;
   	// If resuming, warn when the last recorded model differs from the current one.
   	let curr: &str = turn_context.model_info.slug.as_str();
   	if let Some(prev) = previous_turn_settings
   	.as_ref()
   	.map(|settings| settings.model.as_str())
   	.filter(|model| *model != curr)
   	{
   	warn!("resuming session with different model: previous={prev}, current={curr}");
   	self.send_event(
   	&turn_context,
   	EventMsg::Warning(WarningEvent {
   	message: format!(
   	"This session was recorded with model `{prev}` but is resuming with `{curr}`. \
   	Consider switching back to `{prev}` as it may affect Codex performance."
   	),
   	}),
   	)
   	.await;
   	}
   	// Seed usage info from the recorded rollout so UIs can show token counts
   	// immediately on resume/fork.
   	if let Some(info) = Self::last_token_info_from_rollout(&rollout_items) {
   	let mut state = self.state.lock().await;
   	state.set_token_info(Some(info));
   	}
   	// Defer seeding the session's initial context until the first turn starts so
   	// turn/start overrides can be merged before we write to the rollout.
   	if !is_subagent {
   	let _ = self.flush_rollout().await;
   	}
   	}
   	InitialHistory::Forked(mut rollout_items) => {
   	let turn_context = self.new_default_turn().await;
   	if turn_context.item_ids_enabled() {
   	Self::assign_missing_rollout_response_item_ids(&mut rollout_items);
   	}
   	self.apply_rollout_reconstruction(&turn_context, &rollout_items)
   	.await;
   	// Seed usage info from the recorded rollout so UIs can show token counts
   	// immediately on resume/fork.
   	if let Some(info) = Self::last_token_info_from_rollout(&rollout_items) {
   	let mut state = self.state.lock().await;
   	state.set_token_info(Some(info));
   	}
   	// Paginated subagents persist inherited model context while creating the live
   	// thread so the copied prefix is not observed as child-owned metadata.
   	if !rollout_items.is_empty() && !is_paginated_subagent {
   	self.persist_rollout_items(&rollout_items).await;
   	}
   	// Forked threads should remain file-backed immediately after startup.
   	self.ensure_rollout_materialized().await;
   	// Flush after seeding history and any persisted rollout copy.
   	if !is_subagent {
   	let _ = self.flush_rollout().await;
   	}
   	}

   codex/codex-rs/core/src/session/mod.rs

   Lines 1345 to 1370 in 0fb559f

   	async fn apply_rollout_reconstruction(
   	&self,
   	turn_context: &TurnContext,
   	rollout_items: &[RolloutItem],
   	) -> Option<PreviousTurnSettings> {
   	let rollout_reconstruction::RolloutReconstruction {
   	mut history,
   	previous_turn_settings,
   	reference_context_item,
   	world_state_baseline,
   	window_number,
   	first_window_id,
   	previous_window_id,
   	window_id,
   	} = self
   	.reconstruct_history_from_rollout(turn_context, rollout_items)
   	.await;
   	// Keep the recorded rollout unchanged. Prepare its reconstructed history before
   	// installing it, so legacy media is processed once for this resume or fork and
   	// will be processed again if the rollout is reconstructed in a future session.
   	// This meets media preparation requirements without modifying persisted rollouts.
   	prepare_image_response_items(&mut history);
   	prepare_audio_response_items(&mut history);
   	{
   	let mut state = self.state.lock().await;
   	state.replace_history(history, reference_context_item);

   Proposed implementation / fix spec

   I suggest landing the smallest coherent safety fix first: raw media bytes should not cross a successful compaction boundary, but reopenable source references should. The compaction output is already the semantic handoff for the prior window.

   1. Add a shared helper that replaces retained InputImage and InputAudio parts before retained-message budgeting.
      • For the canonical <image ... path="...">, InputImage, </image> sequence, preserve the existing path wrapper and replace only InputImage with bounded text such as Image bytes removed after compaction; use view_image with the path above if needed.
      • This avoids parsing or rewriting path strings and retains a useful model-callable reference.
      • For media without a durable source reference, use a bounded unavailable marker rather than retaining the data URL.
   2. Apply that helper inside build_v2_compacted_history before truncate_retained_messages_for_remote_compaction.
      • The reference/marker text is then charged against the existing 64k budget.
      • Message order, existing path wrappers, IDs, phases, and passthrough metadata remain intact.
      • The newly installed replacement history contains no raw inline media.
   3. Add a synchronous historic-rollout repair pass at the shared resume/fork reconstruction boundary, before any model request.
      • reconstruct_history_from_rollout already distinguishes the latest base replacement_history from the newer rollout suffix.
      • Sanitize only that base replacement history. Do not sanitize media in the post-checkpoint suffix because it has not yet been summarized.
      • Preserve adjacent local-image path wrappers and replace only raw media parts.
      • Return a repair report containing whether the history changed, omitted counts/bytes, and the repaired checkpoint metadata.
      • Make the transformation deterministic and idempotent.
   4. Persist one durable repaired checkpoint when the reconstruction pass changes history.
      • On resume, append a sanitized replacement-history checkpoint with the same window number/IDs and compaction payload; this is a representation migration, not a new context window.
      • On fork, persist the sanitized checkpoint only in the child rollout and leave the parent byte-for-byte unchanged.
      • Add an explicit rollout migration/media-policy version so a second resume does not append an equivalent checkpoint.
      • Persist the repaired checkpoint before accepting the first user/model turn, so a crash cannot leave in-memory and future-resume semantics divergent.
   5. Keep a defensive recovery pass in run_remote_compact_v2_attempt for already-running sessions upgraded in place or histories that bypass cold reconstruction.
      • The cloned history is available before for_prompt() constructs the compaction request:
        

        codex/codex-rs/core/src/compact_remote_v2_attempt.rs

        Lines 41 to 85 in 0fb559f

        	let mut history = sess.clone_history().await;
        	let base_instructions = sess.get_base_instructions().await;
        	let (rewritten_outputs, estimated_deleted_tokens) =
        	trim_function_call_history_to_fit_context_window(
        	&mut history,
        	turn_context.as_ref(),
        	&base_instructions,
        	);
        	if rewritten_outputs > 0 {
        	info!(
        	turn_id = %turn_context.sub_id,
        	rewritten_outputs,
        	"rewrote history outputs before remote compaction v2"
        	);
        	}
        	if estimated_deleted_tokens > 0 {
        	let max_local_deleted_tokens = sess
        	.estimated_tokens_after_last_model_generated_item()
        	.await;
        	analytics_details.active_context_tokens_before = analytics_details
        	.active_context_tokens_before
        	.map(|active_context_tokens_before| {
        	active_context_tokens_before
        	.saturating_sub(estimated_deleted_tokens.min(max_local_deleted_tokens))
        	});
        	}
        	let trace_input_history = history.raw_items().to_vec();
        	let prompt_input = history.for_prompt(&turn_context.model_info.input_modalities);
        	let tool_router = built_tools(
        	sess.as_ref(),
        	step_context.as_ref(),
        	&CancellationToken::new(),
        	)
        	.await?;
        	let mut input = prompt_input.clone();
        	input.push(ResponseItem::CompactionTrigger {});
        	let prompt = Prompt {
        	input,
        	tools: tool_router.model_visible_specs(),
        	parallel_tool_calls: turn_context.model_info.supports_parallel_tool_calls,
        	base_instructions,
        	output_schema: None,
        	output_schema_strict: true,
        	};

      • Find the most recent ResponseItem::Compaction / ContextCompaction boundary.
      • Replace raw media only in items at or before that boundary because those items were already summarized by the previous checkpoint.
      • Preserve adjacent local-image path wrappers so historical images remain reopenable.
      • Keep media added after the boundary so the current compaction model can inspect and summarize it.
      • Record the sanitized history as the actual trace input so trace data matches the model request.
   6. After the compaction response succeeds, sanitize all retained raw media before installing the next replacement history while keeping available references. This means first-compaction media is visible to the compactor once, but is not replayed indefinitely afterward and can still be reopened on demand.
   7. Treat physical rollout cleanup as a separate non-blocking vacuum after successful reconstruction.
      • The synchronous repair fixes active context but an appended checkpoint does not reclaim bytes from superseded records.
      • Never rewrite a multi-gigabyte rollout on the resume critical path.
      • Under an exclusive rollout-writer lock, stream the source into a same-directory temporary file and strip raw media only from superseded CompactedItem.replacement_history snapshots.
      • Preserve original user/tool response records, ordering, IDs, timestamps, source-path text, and opaque encrypted content byte-for-byte.
      • Validate every output JSONL record, fsync the file and directory, retain a recoverable backup, and use an atomic replacement. Any interruption before the final rename must leave the original usable.
      • Fork cleanup must never rewrite the parent rollout.
   8. Complete the reverse/lazy checkpoint loader already anticipated by the reconstruction comments. Until then, current eager loading must still parse the old large file before the repair pass can run; the repaired checkpoint fixes model context immediately, while the vacuum is what restores current-startup and disk behavior.
   9. Extend existing compaction/reconstruction analytics with omitted_image_count, omitted_audio_count, omitted inline-media bytes, repair-applied/version, vacuum bytes before/after, and vacuum failure status. Keep retained_image_count for verifying that the installed replacement reaches zero under this policy.
   10. Treat environment ownership explicitly.

   • Model-generated view_image paths remain strings at the tool boundary, but resolution should continue through the selected environment and its filesystem.
   • When the source is not in the default environment, retain enough provenance to supply environment_id.
   • Do not assume that a host path is readable by a remote executor.

   11. For pathless pasted/data-URL images, a later stage can materialize content once into an environment-owned, content-addressed media store and retain a stable reference. If that storage contract is not available, the safe first-stage fallback is an unavailable marker, not inline bytes.
   12. If product requirements later demand raw post-compaction media retention, add it as a separately reviewed bounded policy using the existing model-visible estimator plus explicit count/byte caps. It should not remain an accidental zero-cost exception to a text budget.

   Acceptance criteria

   • A first compaction request can include images from the current window so the compaction model can summarize them.
   • The installed replacement_history after compaction contains no raw InputImage or InputAudio payloads.
   • A later compaction request does not replay media that was already covered by the preceding compaction output.
   • A cold resume from a legacy image-heavy checkpoint sanitizes the base replacement history before the first model request while preserving unsummarized suffix media.
   • A resumed thread appends exactly one versioned repaired checkpoint without advancing the context-window identity; a second resume is a no-op.
   • A fork persists sanitized inherited context only in the child and leaves the parent rollout unchanged.
   • Local-image messages preserve their existing source paths, ordering, metadata, and bounded replacement text.
   • A post-compaction model turn can call view_image with a retained path and successfully reload the image when that source remains available.
   • Pathless or missing sources degrade to an explicit unavailable marker without reintroducing inline bytes.
   • The existing image-preservation tests are replaced with assertions that reference/marker text is budgeted and raw media is absent.
   • An integration test performs two compactions around image inputs and verifies that the second request does not contain prior-window data URLs.
   • Remote-executor coverage verifies that a retained non-default-environment reference supplies the correct environment_id; a host-only path is never treated as an executor path.
   • A regression test with 100+ image messages verifies that post-compaction history stays well below the auto-compaction threshold and a short follow-up does not immediately compact again.
   • A vacuum test starts from a multi-checkpoint legacy rollout, verifies substantial byte reclamation, and proves that reconstructed model context and transcript-visible original image events are unchanged.
   • Vacuum interruption/failure tests verify that the original rollout remains valid and selected until the replacement has been fully validated and atomically installed.
   • Encrypted compaction/reasoning fields remain byte-identical through repair and vacuum.
   • InputAudio receives the same treatment so the parallel zero-cost path does not become the next instance of this bug.

   This would address both the immediate context death spiral here and the repeated checkpoint amplification tracked in #23257 without requiring a new persisted-history representation.
4. dorukardahan commented on Jul 23, 2026

   

   dorukardahan

   on Jul 23, 2026 – with ChatGPT Codex Connector

   More actions

   i can confirm this on another affected Codex 0.145.0 session.

   i used a read-only structural scan. i did not inspect or upload prompts, paths, images, tool output, project names, or thread IDs.

   After one successful compaction, the installed replacement history still contained 32 unique PNG input_image items. All had detail: high. The inline payloads used about 18.8 million base64 characters. The existing estimator in Codex maps these images to about 59,000 model-visible image tokens.

   The first real server usage after that compaction reported 104,988 input tokens and 13,056 cached input tokens. This is consistent with a compacted checkpoint that still keeps a large image floor.

   Current main at 808d3c2702ce8eae007c457aa930e7c3b68dd5f6 still gives images and audio zero cost during retained-message selection and then keeps them:

   codex/codex-rs/core/src/compact_remote_v2.rs

   Lines 439 to 556 in 808d3c2

   	fn build_v2_compacted_history(
   	prompt_input: &[ResponseItem],
   	compaction_output: ResponseItem,
   	) -> (Vec<ResponseItem>, usize) {
   	let retained = prompt_input
   	.iter()
   	.filter(|item| is_retained_for_remote_compaction_v2(item))
   	.filter(|item| should_keep_compacted_history_item(item))
   	.cloned()
   	.collect::<Vec<_>>();
   	let mut retained =
   	truncate_retained_messages_for_remote_compaction(retained, RETAINED_MESSAGE_TOKEN_BUDGET);
   	let retained_image_count = retained
   	.iter()
   	.map(retained_input_image_count)
   	.sum::<usize>();
   	retained.push(compaction_output);
   	(retained, retained_image_count)
   	}
   	fn is_retained_for_remote_compaction_v2(item: &ResponseItem) -> bool {
   	let ResponseItem::Message { role, .. } = item else {
   	return false;
   	};
   	matches!(role.as_str(), "user" | "developer" | "system")
   	}
   	fn retained_input_image_count(item: &ResponseItem) -> usize {
   	let ResponseItem::Message { content, .. } = item else {
   	return 0;
   	};
   	content
   	.iter()
   	.filter(|item| matches!(item, ContentItem::InputImage { .. }))
   	.count()
   	}
   	fn truncate_retained_messages_for_remote_compaction(
   	items: Vec<ResponseItem>,
   	max_tokens: usize,
   	) -> Vec<ResponseItem> {
   	let mut remaining = max_tokens;
   	let mut truncated_reversed = Vec::with_capacity(items.len());
   	for item in items.into_iter().rev() {
   	if remaining == 0 {
   	continue;
   	}
   	let token_count = message_text_token_count(&item).max(1);
   	if token_count <= remaining {
   	truncated_reversed.push(item);
   	remaining = remaining.saturating_sub(token_count);
   	} else if let Some(truncated_item) =
   	truncate_message_text_to_token_budget(item, /*max_tokens*/ remaining)
   	{
   	truncated_reversed.push(truncated_item);
   	remaining = 0;
   	}
   	}
   	truncated_reversed.reverse();
   	truncated_reversed
   	}
   	fn message_text_token_count(item: &ResponseItem) -> usize {
   	let ResponseItem::Message { content, .. } = item else {
   	return 0;
   	};
   	content
   	.iter()
   	.map(|item| match item {
   	ContentItem::InputText { text } | ContentItem::OutputText { text } => {
   	approx_token_count(text)
   	}
   	ContentItem::InputImage { .. } | ContentItem::InputAudio { .. } => 0,
   	})
   	.sum()
   	}
   	fn truncate_message_text_to_token_budget(
   	item: ResponseItem,
   	max_tokens: usize,
   	) -> Option<ResponseItem> {
   	let ResponseItem::Message {
   	id,
   	role,
   	content,
   	phase,
   	internal_chat_message_metadata_passthrough: metadata,
   	} = item
   	else {
   	return Some(item);
   	};
   	let mut remaining = max_tokens;
   	let mut truncated_content = Vec::with_capacity(content.len());
   	for mut content_item in content {
   	match &mut content_item {
   	ContentItem::InputText { text } | ContentItem::OutputText { text } => {
   	if remaining == 0 {
   	continue;
   	}
   	let token_count = approx_token_count(text);
   	if token_count <= remaining {
   	remaining = remaining.saturating_sub(token_count);
   	} else {
   	*text = truncate_text(text, TruncationPolicy::Tokens(remaining));
   	remaining = 0;
   	}
   	if !text.is_empty() {
   	truncated_content.push(content_item);
   	}
   	}
   	ContentItem::InputImage { .. } | ContentItem::InputAudio { .. } => {
   	truncated_content.push(content_item);

   The live history is then prepared with for_prompt() before each model request:

   codex/codex-rs/core/src/session/turn.rs

   Lines 292 to 299 in 808d3c2

   	// Construct the input that we will send to the model.
   	let sampling_request_input: Vec<ResponseItem> = async {
   	sess.clone_history()
   	.await
   	.for_prompt(&turn_context.model_info.input_modalities)
   	}
   	.instrument(trace_span!("run_turn.prepare_sampling_request_input"))
   	.await;

   The same codebase estimates an ordinary resized image as about 1,844 tokens:

   codex/codex-rs/core/src/context_manager/history.rs

   Lines 493 to 506 in 808d3c2

   	/// Returns the same coarse, model-visible token estimate used for full history estimates.
   	///
   	/// Ordinary items are JSON-serialized, so callers estimating many items should reuse these
   	/// results instead of repeatedly estimating the full history.
   	pub(crate) fn estimate_item_token_count(item: &ResponseItem) -> i64 {
   	let model_visible_bytes = estimate_response_item_model_visible_bytes(item);
   	approx_tokens_from_byte_count_i64(model_visible_bytes)
   	}
   	/// Approximate model-visible byte cost for one image input.
   	///
   	/// The estimator later converts bytes to tokens using a 4-bytes/token heuristic
   	/// with ceiling division, so 7,373 bytes maps to approximately 1,844 tokens.
   	const RESIZED_IMAGE_BYTES_ESTIMATE: i64 = 7373;

   This does not prove an exact image-only token cost because usage events do not include a modality split. It does confirm a mismatch between retained-message selection and the model-visible estimator.

   i agree with the invariant proposed above. Raw media from the compacted window should not enter the installed replacement history. A bounded reference or unavailable marker can remain, while the original media stays available to the transcript. i am not posting the rollout because it contains private data.

   If maintainers want legacy repair and physical vacuum tracked separately, i can open a focused follow-up issue.
5. romainsimon commented on Jul 23, 2026

   

   romainsimon

   on Jul 23, 2026

   Last edited by romainsimon

   More actions

   Adding a corroborating data point for this on macOS, because the impact went beyond disk and into a very large token bill.

   Environment

   • ChatGPT desktop, version 26.715.72359 (build 5718)
   • Codex framework 150.0.7871.124
   • macOS (Apple Silicon), 460 GB disk

   What happened
   One image-heavy thread (lots of generated images over a long session) grew past 1 GB. From there it matches the mechanism described in this issue exactly: the inline image data gets re-serialized on compaction/resume, so the session is re-written in full again and again instead of referencing prior image content.

   On-disk evidence (~/.codex/sessions)

   • Sessions directory reached ~81 GB.
   • 117 session files were over 200 MB, totaling ~59 GB.
   • Largest single session file: 1.5 GB; several others 700 MB to 1.1 GB.
   • In a single day, in one project, 101 session files exceeded 200 MB across ~81 distinct session ids, totaling ~46 GB. Many were near-identical in size and created seconds to minutes apart (e.g. four ~826 MB files within an 11-minute window), i.e. repeated full copies of the same conversation rather than distinct sessions.
   • By contrast, 451 normal (text-only) session files were under 50 MB and totaled ~3 GB combined.

   The disk (460 GB) filled to 0 bytes twice in one day, making the machine unusable (writes failing with ENOSPC system-wide). Multiple Codex renderer processes sat at 70 to 75% CPU while this happened.

   Token / cost impact (this is the part I'd flag)
   Because each resume re-feeds the entire ~1 GB session as context, usage tracking (ccusage) reported roughly 2.68 billion input tokens and ~73 billion cached input tokens for Codex/gpt-5 in a single day, estimated at about $35,700 for that one day!!! Non-Codex usage on the same machine that day was under $100.

   I suspect this is #33735 (giant sessions from duplicated inline images) compounding with #22411 (app-server eager-loading all session files on every thread/list): once each session is multiple GB, repeatedly re-reading them all turns the disk bloat into a massive background token burn as well. So the two issues together produce a much worse outcome than either alone.

   Suggested fixes (echoing and extending this thread)

   • Store images out of line as separate blobs referenced by id, never inline base64 in the rollout, and never re-embed them in compaction/replacement_history records.
   • On resume/compaction, append or reference rather than writing a full duplicate copy of the whole session.
   • Add a retention/size cap and garbage-collection for old rollout copies, with a warning when a session grows abnormally large.

   Workaround
   Fully quit the ChatGPT/Codex app (so no session files are held open), then delete the redundant oversized rollout copies from ~/.codex/sessions while preserving the normal small threads. That reclaimed ~50 GB in my case.
6. bbingz commented on Jul 25, 2026

   

   bbingz

   on Jul 25, 2026

   More actions

   Analysis (community)

   Residual analysis for #33493 (compaction v2 retains unbounded input_image payloads → repeated auto-compaction) under docs/contributing.md — invitation-only. No unsolicited PR.

   Root-cause hypothesis

   The images are only the trigger. The reason the cycle is relentless rather than occasional is a missing fixed-point guarantee in the compaction loop: nothing ever checks whether the post-compaction history actually fits below the auto-compact threshold, so any retained payload (images counted at 0 tokens, an uncapped chained summary) keeps the prompt at ~97% of the limit and re-fires compaction on every subsequent turn — while each fire appends another full replacement_history snapshot to an append-only rollout with no GC. Verified at HEAD 4c43465133:

   1. Images are counted as 0 tokens and budget-exempt — exactly as reported. In the v2 truncation pass, message_text_token_count maps ContentItem::InputImage and InputAudio to 0 (codex-rs/core/src/compact_remote_v2.rs:505-525), and truncate_retained_messages_for_remote_compaction (compact_remote_v2.rs:478-502) only ever truncates text via truncate_message_text_to_token_budget — an image-only message sails through the 64k RETAINED_MESSAGE_TOKEN_BUDGET (:56) untouched. The local path has no image budget at all. Your 92 retained images / 26.47M base64 chars in the newest compacted record is this code path.

   2. No post-compaction fits-check → no fixed point. After compaction, recompute_token_usage only re-estimates (compact.rs:384); nothing compares the result against auto_compact_token_limit() (= context_window * 9/10, protocol/src/openai_models.rs:459-470). Pre-turn compaction fires on every turn when token_limit_reached (session/turn.rs:846-872), and the mid-turn rollover path carries the dev comment "as long as compaction works well in getting us way below the token limit, we shouldn't worry about being in an infinite loop" (turn.rs:394). With ~8k headroom after compaction (your measurement), one normal follow-up turn crosses 90% again — the loop has no exit condition. There is also no re-fire guard: nothing records "just compacted, usage unchanged" (state/session.rs has no such marker).

   3. The retained summary is uncapped and chained. Each compaction builds replacement history from ≤20k tokens of recent user messages (COMPACT_USER_MESSAGE_MAX_TOKENS, compact.rs:56) plus SUMMARY_PREFIX + summary_suffix where the suffix is the last assistant message of the compaction turn (compact.rs:347-353) — i.e. the prior summary re-summarized, appended with no size cap in build_compacted_history (compact.rs:671-684). Text alone can therefore hold the history near threshold across compactions even after the images are fixed.

   4. Every compaction appends a full snapshot; the chain is never pruned. replace_compacted_history persists RolloutItem::Compacted carrying the complete replacement_history, plus full WorldState and TurnContext items (session/mod.rs:3168-3215), into an append-only writer. Reconstruction consumes only the newest surviving checkpoint (session/rollout_reconstruction.rs:156-186 — the reverse scan takes the first replacement_history it meets and replays only the suffix). Every older snapshot is provably dead weight, and there is no GC/retention pass anywhere in rollout/ or core/. Your 4.16 GB / 224 compaction records — and the closed #24191's 892 MB / 62 records — are this mechanic; image payloads just make each record ~100x larger.

   5. The overflow safety valve is itself unbounded. On ContextWindowExceeded the local path removes ONE oldest history item, resets retries = 0, and re-issues a full remote stream call (compact.rs:309-324) — an item-count-bounded retry loop with no constant cap; the remote path treats it as non-retryable (protocol/src/error.rs:358-396), which is the terminal context_length_exceeded sibling of this issue.

   Ranked fix outline (all OSS, codex-rs):

   • (a) Count input_image at a real token estimate (or a conservative per-image constant) in message_text_token_count, and let truncate_retained_messages_for_remote_compaction drop/placeholder oldest images when over budget — closes the reported trigger inside the code the v2 fix introduced.
   • (b) Post-compaction fits-check: after recompute_token_usage, if usage is still ≥ the auto-compact limit, escalate deterministically (drop injected initial context → halve retained user-message budget → cap summary) and, failing that, emit an actionable terminal error instead of silently re-arming the same loop. This replaces the turn.rs:394 assumption with a structural guarantee.
   • (c) Cap the chained summary (e.g. 4–8k tokens) in build_compacted_history.
   • (d) Re-fire guard: skip PreTurn auto-compact when the last compaction was in the current window, no new turn occurred, and usage hasn't grown past the post-compaction baseline.
   • (e) Checkpoint-chain GC: since reconstruction only ever reads the newest Compacted, a best-effort pruner (temp-file + rename, modeled on the existing cold-compression worker in rollout/src/compression.rs) keeping SessionMeta + newest checkpoint + suffix bounds on-disk growth to O(window) instead of O(total tokens). Largest diff; also relieves the rollout-size half of this issue.
   • (f) Cap the trim-1-retry loop with a constant MAX_TRIM_PASSES and bail with an actionable error before reaching 1 item.

   Fail-first tests I'd want before any merge: (1) a retained history with 26M chars of input_image and <1k tokens of text is truncated by the v2 budget pass (today: passes through intact); (2) after a compaction whose recomputed usage is still ≥ threshold, the next turn does NOT immediately re-fire an identical PreTurn compaction (today: it does); (3) K compactions leave exactly 1 live Compacted snapshot plus suffix in the rollout (today: K full snapshots). Happy to draft the test patch if a maintainer wants it.

   Related: #32888 is the stale-token-usage sibling on the same resume/trigger axis (resume seeds live usage from the last persisted TokenCount, session/mod.rs:1340-1345); #33235 was auto-flagged as a possible duplicate. No unsolicited PR from me.
7. fujiwarakasei commented on Jul 29, 2026

   

   fujiwarakasei

   on Jul 29, 2026 – with ChatGPT Codex Connector

   More actions

   Adding a sanitized reproduction from a newer macOS/App/CLI combination. This corroborates the existing diagnosis and quantifies the on-disk amplification from repeated inline images.

   Environment

   • ChatGPT/Codex Desktop: 26.721.41059 (build 5848)
   • Affected rollout metadata: cli_version 0.145.0-alpha.18
   • Currently installed CLI: codex-cli 0.146.0
   • macOS: 26.5.2 (25F84), Apple Silicon
   • Model: gpt-5.6-sol

   Sanitized structural measurements

   The affected rollout was inspected with a streaming structural analyzer. No prompts, screenshots, paths, credentials, or raw records are attached.

   Metric	Observed value
   Rollout JSONL size	1,905,021,000 bytes (1.774 GiB)
   JSONL records	8,068
   compacted records	51
   Bytes in compacted records	1,751,228,762 (91.93% of file)
   Inline data:image references	3,426
   Bytes in inline image strings	1,872,392,983 (98.29% of file)
   JPEG / PNG references	3,403 / 23
   Exact unique image data URLs	331
   Bytes in one copy of exact unique image data URLs	116,216,991 (110.83 MiB)
   Weighted repetition factor	16.11x
   Repeated inline-image excess above one exact copy	1.636 GiB
   Non-image JSON/text/metadata remainder	about 31.1 MiB
   Inline audio / other base64 data URLs	0 / 0

   The largest single compacted record was 39,309,704 bytes. It contained 69 inline image references totaling 39,140,726 bytes; 68 were unique within that checkpoint. Several later checkpoints had effectively the same inline-image byte total, showing that the same retained media was serialized again into successive replacement_history snapshots.

   The 331 exact unique image data URLs represent about 83.1 MiB of decoded image bytes, but repeated embedding plus base64 expansion turned them into roughly 1.74 GiB of text in the rollout. Removing image strings from the accounting leaves only about 31 MiB for all ordinary JSON, text, and metadata.

   This confirms that the abnormal growth is not ordinary conversation text. It is dominated by repeated full input_image data URLs in append-only compaction checkpoints, and the behavior persists on the version combination above.

   The raw rollout is private and will not be uploaded, but I can provide additional redacted structural counters if maintainers need a specific measurement.
8. 24 remaining items

   Load more

9. Klohto commented on Aug 18, 2026

   

   Klohto

   on Aug 18, 2026 – with ChatGPT Codex Connector

   More actions

   I can reproduce this on the current macOS desktop build, with slow compaction as the main symptom.

   Environment

   • Codex Desktop app: 26.810.52044
   • Codex CLI: 0.147.0
   • macOS: 26.5 (25F71)
   • Model: gpt-5.6-sol, high reasoning
   • Affected task ID: 01a01430-d0a2-7c03-a278-a1d7d201e696

   Redacted diagnostic evidence

   • The latest auto-compaction began at 2026-08-18T17:14:33Z and completed at 17:17:39Z: 3 minutes 6 seconds.
   • The request ran through run_auto_compact{reason=ContextLimit phase=MidTurn}.
   • Context immediately before compaction was 251,651 / 258,400 tokens.
   • The logs show a normal Responses WebSocket compaction request. There was no retry, authentication failure, or transport error during this compaction.
   • The newest compacted replacement_history is 5,219,050 serialized bytes with 57 retained items.
   • It retains 10 input_image values containing 5,022,440 inline image/data-URL characters.
   • Three retained user messages account for almost the entire payload: approximately 1.60 MB, 2.18 MB, and 1.25 MB.
   • Compacted replacement-history size grew across the task from 14,507 bytes to 1,756,504, 3,956,864, 5,212,655, and finally 5,219,050 bytes.
   • Recent compactions take about 2:45 to 5:32. The latest took 3:06.
   • The append-only rollout is currently 57,795,192 bytes.

   No task text, screenshots, raw rollout, tool output, local paths, or credentials are attached.

   This corroborates the reported cause. Compaction preserves full historical input_image payloads, so later compactions repeatedly process a multi-megabyte multimodal history. In this case, the request succeeds but the latency has become several minutes.
10. kevindh19 commented on Aug 19, 2026

    

    kevindh19

    on Aug 19, 2026 – with ChatGPT Codex Connector

    More actions

    Additional current macOS reproduction: rapid compaction loop

    I reproduced the same retained-input_image failure on a separate current Codex Desktop task.

    Environment

    • Codex Desktop origin
    • Embedded Codex CLI: 0.148.0-alpha.15
    • macOS 26.5.1, Apple Silicon
    • Model context window reported by the rollout: 258,400 tokens

    Privacy-safe structural diagnostics

    • Task created less than one day before the failure.
    • Rollout JSONL reached 885,073,090 bytes.
    • The task contained 46 image inputs.
    • It had 29 compaction records at measurement time.
    • Compaction records alone accounted for approximately 831.9 MB of the rollout.
    • Each recent compacted payload was approximately 33.6 MB and retained all 46 images.
    • The task entered a burst of 9 compactions within five minutes, including compactions separated by roughly 20–35 seconds.
    • A real request immediately before compaction reported approximately 246,520 / 258,400 tokens.
    • The provisional post-compaction state remained around 210,078 / 258,400 tokens, leaving the task close enough to the threshold that another ordinary tool/model step immediately triggered compaction again.

    No prompts, screenshots, raw rollout data, private project content, local paths, credentials, or task identifiers are included.

    This independently corroborates both reported effects: compacted history retains the full historical image set instead of bounding or externalizing it, and every new checkpoint duplicates those image payloads into the append-only rollout. The resulting task remained technically active but became operationally unusable because compaction no longer created meaningful headroom.

    I can provide the privacy-safe jq measurement commands or help design a regression fixture if maintainers want them, and I would be willing to implement a scoped fix if invited under the repository contribution policy.
11. bbingz commented on Aug 20, 2026

    

    bbingz

    on Aug 20, 2026

    More actions

    Analysis (community)

    @kevindh19 — thanks for the current macOS Desktop reproduction (embedded CLI 0.148.0-alpha.15). That still matches the residual I traced earlier on #33493.

    Root-cause hypothesis (re-checked 2026-08-20 at main 3b45c2906): remote compaction v2 still treats inline input_image as zero tokens for the retained-history budget, then keeps the image bytes when truncating text. So a successful compact does not create a fixed point: the next turn can still be over budget and compact again.

    Live path:

    1. message_text_token_count counts InputText/OutputText only; ContentItem::InputImage and InputAudio contribute 0 (codex-rs/core/src/compact_remote_v2.rs:634-647).
    2. truncate_message_text_to_token_budget truncates text against remaining, but the InputImage/InputAudio arm unconditionally pushes the content item (:689-691) — images are never dropped by the token budget.
    3. retained_input_image_count exists (:545-552) as a metric, not as a drop/GC gate.

    That is why a current Desktop task can still enter a rapid compaction loop after images are in replacement_history: compact shrinks text, images stay, usage stays high, compact fires again.

    This is distinct from #33235 (image inherit on fork into model context). Same payload class, different copy event (compaction retain vs fork inherit).

    No unsolicited PR from me. If maintainers want a fail-first test, the existing build_v2_compacted_history_counts_retained_input_images (:914) is the right fixture to extend: after compact, retained images must either count toward the budget or be stripped.

    Happy to refine the HEAD pin if a newer compact path landed after 3b45c2906.
12. markmdev commented on Aug 21, 2026

    

    markmdev

    on Aug 21, 2026

    More actions

    Current macOS build still reproduces this, with per-turn latency

    This is still happening on the current installed Codex Desktop build. It is not only damage left behind by an older build.

    Environment:

    • Codex Desktop: 26.818.22352 (build 6872)
    • Embedded CLI: 0.148.0-alpha.21
    • macOS 26.6.1 (25G76), Apple Silicon
    • Affected task started on 2026-08-19 with embedded CLI 0.148.0-alpha.15

    The app updated and restarted on August 20. After that restart, the affected task produced two more compacted records, at 21:52:58Z and 23:21:09Z. Each retained the same nine input_image items inside payload.replacement_history, totaling 16,755,582 inline image characters. This confirms that the current build is still creating the bad state.

    A privacy-safe structural scan of the task now shows:

    • rollout size: 205,621,888 bytes
    • JSONL records: 5,018
    • completed compactions: 7
    • input_image nodes retained across completed replacement histories: 40
    • retained compacted-image payload: 63,405,544 characters
    • images outside completed compaction history: 43 (126,975,157 characters), left untouched by the scan

    The user-visible symptom is latency on every turn, not only during compaction. New tasks on the same installation respond in roughly 1–2 seconds, while this task takes 40+ seconds for trivial prompts. A recent short acknowledgement took 68.493s; another ordinary turn took 195.155s.

    A schema-aware dry run found that the affected task can be repaired by replacing only input_image nodes inside completed payload.replacement_history, while preserving current/uncompacted images and backing up the rollout. No cleanup was applied before taking these measurements.

    The product needs both parts:

    1. Stop retaining inline image payloads in new compacted replacement histories.
    2. Provide a safe migration or built-in repair action for tasks already affected.

    I am not attaching the rollout because it contains private prompts, screenshots, local paths, and tool output.
13. laffo16 commented on Aug 21, 2026

    

    laffo16

    on Aug 21, 2026

    More actions

    I have a second reproduction matching this issue, this time in Codex CLI on Windows.

    Environment

    • Platform: Windows 10.0.19044 x64
    • Terminal: Windows Terminal / PowerShell 7.6.5
    • Model: gpt-5.6-sol
    • Reasoning effort: max
    • Summaries: auto
    • Collaboration: multi-agent v2, default mode
    • Subscription: ChatGPT Pro
    • Effective context window: 258,400
    • Session ID: 01a01907-f597-7d91-a8c1-e7cb0ce7d5d0
    • Session creation metadata records CLI 0.148.0
    • The current installation and /status report CLI 0.149.0 for the same resumed session. The rollout does not record the client version for every resume.

    User-visible behavior

    The TUI repeatedly displayed Context compacted without completing useful work:

    • One screenshot shows three consecutive compactions while still Working after 6m33s.
    • A subsequent screenshot shows four consecutive compactions while still Working after 6m42s.
    • The UI later displayed Conversation interrupted.
    • /status then showed 0% left (285K used / 258K) around another compaction.

    

    Privacy-safe rollout measurements

    These measurements inspect record structure, counts and sizes only. I am not attaching the raw rollout because it contains private prompts, paths, tool output and images.

    Measured at compaction 60 on 2026-08-21T16:21:45Z:

    Metric	Observed value
    Total compactions	60
    Rollout JSONL size	667.23 MiB
    Size occupied by compaction records	543.48 MiB (81.5%)
    Latest compacted record	12.13 MiB
    Latest replacement_history	317 items
    Retained input_image nodes	131
    Distinct exact image payloads in latest history	130
    Retained image data-URL characters	12,502,222
    Retained text characters	102,782
    Image share of retained image + text characters	approximately 99.2%
    Image occurrences across all 60 compaction snapshots	5,706
    Distinct exact image payloads across those snapshots	130
    Average reserialization per distinct image	43.9 times
    Image MIME type	all image/png

    This is not mainly duplicate images within a single history: the latest history has 131 image nodes representing 130 exact payloads. The major amplification is the same retained images being copied into each subsequent compacted record.

    Rapid compaction bursts

    Two separate loops occurred:

    1. Compactions 42-51: 10 compactions in 8m09s.
    2. Compactions 52-60: 9 compactions in 7m22s.

    There were zero function / tool calls between the compactions in either burst. Individual intervals were often only about 32-72 seconds.

    The nearest token telemetry around compaction 42 changed from 226,678 before compaction to 295,687 after it. Subsequent post-compaction totals remained around 279K-296K.

    For compaction 60:

    • Before: 299,346
    • After: 299,414
    • Context window: 258,400

    Across the rollout, 31 of 60 first post-compaction token samples were already at or above the full 258,400 context window. This explains why compaction immediately re-armed without a meaningful new turn.

    The /status UI and rollout telemetry use somewhat different totals (285K versus 299,414 in the latest observation), but both independently place the session above the 258K window immediately after compaction.

    Multi-agent context

    The run spawned seven subagents and contained substantial collaboration activity before the loop. That may have accelerated history growth, but the actual rapid-compaction windows contained no agent or other tool calls, so multi-agent activity does not appear to be the immediate loop mechanism.

    The rollout also contains six turn_aborted events overall, and the TUI displayed Conversation interrupted during the later burst.

    Interpretation

    This appears to reproduce the mechanism described in this issue:

    1. Compaction retains the full historical input_image data URLs.
    2. Images dominate the resulting replacement history.
    3. The post-compaction state can remain above the entire context window.
    4. Auto-compaction therefore starts again without meaningful intervening work.
    5. Every compaction persists another approximately 12 MiB copy of substantially the same image history.

    The separate reasoning-accounting issue in #39767 may influence when the first compaction is triggered on gpt-5.6-sol, but it cannot account for the retained 12.5 million image characters or the repeated copying of 130 image payloads. This rollout therefore provides independent evidence for the retained-image problem reported here.

    I can provide additional redacted structural measurements if maintainers need them.
14. garpunzel commented on Aug 21, 2026

    

    garpunzel

    on Aug 21, 2026

    More actions

    Additional current macOS reproduction: gradual image accumulation causes per-turn compaction thrash

    Privacy-safe structural scan of another affected Codex Desktop rollout. This case accumulated screenshots over a long-running thread rather than attaching one large batch.

    Environment

    • Codex Desktop origin
    • Rollout creation metadata: embedded CLI 0.148.0-alpha.9
    • Model: gpt-5.6-sol, xhigh
    • Effective context window: 353,400 tokens

    Structural measurements

    • Completed compaction records: 18
    • Retained input_image nodes in latest replacement history: 29
    • Retained inline image/data-URL characters: 49,162,318
    • Latest replacement history: 124 items, approximately 49,367,092 serialized characters
    • Compaction windows 17 and 18 retained the exact same ordered image payload set
    • Those two context_compacted events occurred 5m46s apart inside one unfinished turn

    Token sequence

    1. First compaction provisional total: 247,329.
    2. First real post-compaction model call: 309,488 / 353,400 input tokens.
    3. Normal follow-up work grew the prompt to 330,134 input tokens.
    4. Second compaction provisional total: 268,560.
    5. First real post-second-compaction call: 309,619 input tokens.

    The second compaction therefore produced only about 20.5K net prompt reduction relative to the last pre-compaction call. The retained image floor immediately restored the prompt to roughly 87.6% of the effective window.

    No rollback or retry duplicated the UI event: the rollout records distinct compaction window IDs and incremented window_number from 17 to 18.

    This matches the reported mechanism: raw historical images survive the retained-message budget, compaction removes mostly text/tool history, and the next ordinary turn immediately approaches the threshold again. User-visible result is several minutes of compaction latency per question and an operationally unusable task.

    No raw rollout, prompts, paths, images, tool outputs, or thread IDs are included.
15. ccomaeng commented on Aug 22, 2026

    

    ccomaeng

    on Aug 22, 2026

    More actions

    Independent macOS/Desktop reproduction on a current build, with two separate rollouts reaching 25+ GB.

    Environment

    • ChatGPT/Codex Desktop: 26.818.31338 (build 6892)
    • Standalone CLI: codex-cli 0.146.0
    • macOS: 26.6.2 (25G83)
    • Model: gpt-5.6-sol
    • Current upstream source checked at openai/codex@4f39251a010a8bd7d692d25fb33832ff06f1635a

    Privacy-safe structural evidence

    Affected rollout A:

    • File size: 28,091,832,834 bytes
    • compaction records: 261
    • context_compacted events: 261
    • input_image occurrences: 14,743
    • Inline JPEG data URLs: 14,741
    • Retained runtime logs show 141 repeated requests with approximately 108,795,437 bytes before compaction and 80,273,208 bytes after compaction.
    • Observed usage remained approximately 581,308 tokens against an auto-compaction limit of 244,800.
    • 70 observed post-sampling checks reported token_limit_reached=true, repeatedly followed by run_auto_compact with reason=ContextLimit and phase=MidTurn.

    Affected rollout B, from a separate earlier image-heavy ebook task on the same machine:

    • File size: 26,467,249,845 bytes
    • Retained runtime logs show 105 repeated request bodies around 37,991,677 bytes and 51 token-limit events around 245k-251k tokens against the same 244,800 limit.
    • It exhibited the same repeated ContextLimit / MidTurn auto-compaction pattern.

    No raw rollout, prompts, paths, images, or base64 payloads are attached because they contain private data.

    Current source trace

    Current main still matches the observed failure mode:

    • codex-rs/core/src/compact_remote_v2.rs::message_text_token_count charges InputImage and InputAudio as zero text tokens.
    • truncate_message_text_to_token_budget preserves those media content items.
    • build_v2_compacted_history retains user messages and reports the retained image count.
    • codex-rs/core/src/session/turn.rs runs mid-turn auto-compaction and immediately continues the turn. There is no local consecutive-attempt ceiling or check that the compacted context actually fell materially below the threshold before another pass.

    This makes the failure self-amplifying: images survive compaction, the resulting context stays over the limit, another full image-bearing request/history is persisted, and the append-only rollout grows roughly by the repeated payload size on each pass.

    Suggested safety fix

    1. Budget retained images by realistic vision-token cost, encoded bytes, and image count rather than a near-zero message floor.
    2. Replace older reopenable local images with lightweight path/content-addressed references at a successful compaction boundary.
    3. Add a progress invariant and a hard ceiling for consecutive auto-compactions in one turn. If post-compaction context does not fall sufficiently, stop and surface a recoverable warning instead of looping.
    4. Add a rollout byte-growth guard and regression coverage proving that mixed text/image history cannot produce repeated non-decreasing compactions.

    This also corroborates the storage amplification reported in #33735, but the non-decreasing mid-turn compaction loop appears to be the immediate trigger for the two 25+ GB files above.

    ---

    This issue or PR was generated by LazyCodex.
    Tag: lazycodex-generated
16. iwasrobbed commented on Aug 23, 2026

    

    iwasrobbed

    on Aug 23, 2026

    More actions

    FYI Tibo finally posted about this bug: https://x.com/thsottiaux/status/2091407991736332689?s=20

    👍React with 👍2Reacted by laffo16 and garpunzel
17. garpunzel commented on Aug 23, 2026

    

    garpunzel

    on Aug 23, 2026

    More actions

    FYI Tibo finally posted about this bug: https://x.com/thsottiaux/status/2091407991736332689?s=20

    “Dude, we're like on TV right now!”
18. Nurozen commented on Sep 27, 2026

    

    Nurozen

    on Sep 27, 2026

    More actions

    Adding a data point from codex-cli 0.157.0 with compaction_image_budget enabled, since the
    measurements in this thread predate that flag becoming default-on (#40994, 2026-08-26).

    codex features list on the affected install:

    compaction_image_budget    stable    true
    resize_all_images          removed   true
    unified_image_budget       under development   false
    

    The budget is on and the ratchet still happens.

    Measurements

    One thread, 12 compactions, measuring payload.replacement_history on each compacted rollout record:

    window	items	images	base64 MB	text MB
    1	20	3	1.25	0.07
    5	23	3	1.25	0.10
    8	25	3	1.25	0.09
    9	29	6	4.61	0.08
    11	39	7	5.82	0.09
    12	41	7	5.82	0.10

    Text stays flat at 0.07-0.10 MB across all 12. Base64 goes 1.25 → 4.61 → 5.82 MB and never
    decreases. Effective floor on every request in the thread became 5.82 MB that compaction could
    not reduce.

    Where the bytes sit

    The summary itself is not the problem. In window 12, replacement_history is 41 items: 37 user
    messages, 3 developer messages, and one {"type":"compaction","id":"cmp_...","encrypted_content":...}
    holding the actual summary. All 5.82 MB is in the retained user messages, as input_image
    content items with inline data:image/png;base64,... URIs at "detail":"high".

    Distribution in that window:

    2.02MB  0.71MB  0.64MB   (three images, one message)
    1.20MB
    0.57MB  0.57MB           (two images, one message)
    0.11MB
    

    So truncate_retained_messages is keeping every image in the retained set rather than dropping
    them under the image budget.

    Resizing is working, format is not

    resize_all_images is doing its job: a source attachment of 2852x1892 reached the wire at
    1928x1280. But all seven images go up as PNG. Re-encoding those same seven at JPEG q85:

                PNG     JPEG85
    binary     4.36M     1.63M    2.6x
    base64     5.80M     2.17M
    

    Worth noting as a mitigation, though it only moves the ceiling rather than removing it, since the
    retention behavior is unchanged.

    Related

    #41338 explains why the budget doesn't catch this from the model's side: an inline image is ~230
    tokens but megabytes on the wire, so anything reasoning in tokens is blind to it. The ImageReference
    / file-id work in #45543 and #45794 looks like the structural fix.

    Happy to supply the redacted rollout records if useful.
19. H234598 commented on Oct 7, 2026

    

    H234598

    on Oct 7, 2026 – with ChatGPT Codex Connector

    More actions

    Additional extreme persisted-history data point from Linux. For the figures below, I extracted only structural fields, sizes, timestamps, token counters, and hashes; no prompt text, image contents, local paths, or thread identifier are included.

    Version caveat

    The affected rollout was created with codex-cli 0.147.0 and accumulated across later resumes. I inspected it with the current embedded codex-cli 0.161.0, where:

    compaction_image_budget          stable             true
    guardian_reuse_parent_compaction stable             true
    remote_compaction_v2             removed            false
    unified_image_budget             under development  false
    

    Therefore, the figures below are strong evidence for the historical persistence-amplification and compaction fixed-point problem, but they should not be read as a claim that all 1,700 records were generated by 0.161.0. A reopen attempt under 0.161.0 surfaced Context compacted, then the turn was aborted after about 38 seconds and no new successful compacted item was written. Because that attempt did not complete, it does not establish whether the current image budget can recover this persisted history.

    Full-rollout structural scan

    Metric	Observed value
    Rollout JSONL size	9,785,315,290 bytes
    JSONL records / malformed records	526,736 / 0
    compacted records	1,700
    Bytes occupied by compacted records	8,047,639,374 bytes (82.24% of rollout)
    Distinct inline image payload identifiers across all compacted histories (first 16 SHA-256 hex characters)	5
    Summed inline image-string characters across all compacted histories	7,390,381,154
    Occurrences per payload identifier	1,591–1,635

    The latest replacement_history contains 530 items: 528 user messages, one developer message, and one compaction item. It retains the same five images, whose inline strings total 4,620,786 characters. The final compacted JSONL record alone is 5,030,887 bytes. Across the last 24 records, record size remains 4,989,798–5,031,815 bytes (median 5,026,940), so compaction is not converging toward a materially smaller persisted state.

    Fixed-point failure: local estimate versus next real request

    For compaction windows 1677–1700:

    • 24 compactions completed in 1,478.809 seconds.
    • Successive compactions were only 47.570–85.788 seconds apart; median 62.754 seconds.
    • Immediately after each compaction, the local token_count event reported last_token_usage.input_tokens=0 and estimated only 43,503–53,432 total tokens.
    • The next real model call reported 284,914–286,496 input tokens in all 24 cases.
    • The recorded model context window was 258,400 tokens, so every next real call was already above it.

    The measurements establish a repeated estimator/request mismatch plus severe persistence amplification. The likely mechanism is:

    1. The post-compaction local estimate is roughly 230k tokens below the next real request. Image cost missing from that estimate is consistent with the retained payload evidence and this issue's reported code path, but the rollout alone does not prove the exact estimator branch responsible.
    2. In this historical run, no effective fixed-point/no-progress stop interrupted the cycle: 24 further compactions completed within about 25 minutes while every next real request remained above the context window. Across all 1,700 records, each of the five payload identifiers recurs 1,591–1,635 times, producing 7.39 billion repeated inline-image characters.

    Read-only integrity check

    I left the affected thread closed and collected 160 samples over 904 seconds at 5–9-second intervals. Rollout size and mtime did not change, and no sample found an open file descriptor to it. This confirms the measurement process did not resume or further inflate the affected session; it does not claim continuous FD observation.

    This data point suggests two regression checks: one for newly generated histories with compaction_image_budget enabled, and one for legacy persisted histories created before or across that mitigation:

    1. Repeated compaction of a history containing a small, fixed number of large inline images must keep both retained media cost and persisted replacement bytes bounded.
    2. A post-compaction fits-check must use a cost model consistent with the next request, or stop after one no-progress attempt with a recoverable error instead of immediately compacting again.

    Is migration/recovery of already-persisted image-bearing replacement histories intended to be covered by compaction_image_budget, or does that require a separate recovery path?
20. nathanchillz commented on Oct 7, 2026

    

    nathanchillz

    on Oct 7, 2026

    More actions

    Additional macOS reproduction on a newer desktop build, with measured request sizes and long WebSocket stalls. Related symptom: #33094.

    Environment: macOS 26.5.1, 24 GB RAM; desktop 26.1002.52244 (13536); bundled CLI 0.162.0-alpha.2; VS Code extension openai.chatgpt 26.1002.51308; model gpt-6-astra, reasoning effort xhigh. Observed on 2026-10-07.

    After two successful manual compactions, the affected desktop chat retained 31 inline input_image items. Their encoded strings totaled 60,209,084 characters, almost all of the 60,259,416-byte serialized replacement history.

    A minimal diagnostic request took 46.426 seconds in the affected chat, versus 5.784 seconds in a fresh chat with the same model and effort. Working directories differed, so this is not a fully controlled comparison. Local tool execution in the old chat took approximately 71.7 ms.

    The old-chat test made two HTTP requests of 44,504,214 and 44,505,354 bytes after compression. Compression-to-HTTP-200 intervals were 20.503 and 19.998 seconds. These timings include upload and server wait; their respective contributions are not isolated. An earlier compaction had logged WebSocket failures and a fallback to HTTP. Separate VS Code requests measured approximately 23.26 MB compressed each.

    After restarting the desktop app, another minimal diagnostic request remained unanswered for 370.172 seconds, until manually interrupted. It produced no assistant response or tool call. Five retries logged: "websocket closed by server before response.completed". A subsequent WebSocket connection succeeded, but no response arrived before cancellation. No HTTP fallback was logged during this final turn.

    The user reports the same symptoms over Wi-Fi and a mobile hotspot. An independent upload test measured 16.1 Mbit/s. This can contribute to large HTTP request latency, but does not by itself explain the repeated WebSocket failures. Updates, restarts and compactions did not resolve the issue. The exact cause of the connection closures remains unconfirmed.

    A Feedback submission from the affected chat was successfully uploaded. Its session ID has been provided privately through OpenAI Support. No private prompts, images, local paths, credentials or session identifiers are included in this public comment.

    Please investigate whether this reproduction shares the retained-image mechanism described here, and provide a supported repair or migration for existing chats that preserves their history and image references.
21. nathanchillz commented on Oct 7, 2026

    

    nathanchillz

    on Oct 7, 2026

    More actions

    Follow-up: VS Code comparison isolates repeated connection failures on a trivial diagnostic turn.

    On 2026-10-08 (Europe/Paris), I ran the same minimal diagnostic request in an affected existing chat and then in a fresh chat, approximately 3 minutes apart. Both used the same project directory, gpt-6-astra, reasoning effort xhigh, and the same running VS Code Codex process (verified from the transport logs). The submitted text matched after trimming surrounding whitespace.

    The request only asked for a short acknowledgement, one terminal call to list the current directory and its immediate entries, and a short final acknowledgement. No file contents were read or changed, and no images were attached, generated, or requested in either test turn.

    Measurement	Existing chat	Fresh chat
    Total turn duration	64.473 s	6.944 s
    First assistant message recorded	52.356 s	3.070 s
    Local tool call-to-result interval	0.190 s	0.069 s
    WebSocket retry warnings	5 (1/5 through 5/5)	0
    HTTP fallback	After 41.152 s	None observed

    The existing chat repeatedly logged websocket closed by server before response.completed. After falling back to HTTP, it prepared two compressed request bodies of 23,272,677 and 23,272,280 bytes. The fresh chat completed over WebSocket without observed reconnects or fallback. Message timings above are transcript recording times, not exact UI first-token measurements; body sizes are logged compressed bodies, not an independent packet capture.

    This demonstrates a transport/retry failure mode before a trivial local operation, beyond simply waiting for image generation or a complex coding task. Existing history still contains images, so this comparison does NOT rule out retained media or payload size as a trigger, nor determine whether the underlying fault is in the client, server, or an intermediary. It should not be interpreted as proof that images are irrelevant, or that image count alone is the root cause.

    The severe degradation appeared within the last two days; I previously worked with similarly large or larger image histories without this behavior. That onset and comparison are user observations, not a measured historical baseline. The impact spans desktop and VS Code workflows. Starting a fresh chat restores speed in this test, but leaves the existing working history and image references behind.

    Please investigate why affected existing conversations enter repeated WebSocket failures and large HTTP replays, and provide a supported recovery that preserves their history and image references. If this transport symptom should be tracked separately from #33493, please advise. Updates, restarts and prior compactions have already been tried, as described in my earlier comment.

    Only sanitized timings, byte counts, settings and the generic error string are shared here. No private conversation content, images, project names, local paths, process IDs or session identifiers are included.
22. dreampiggy commented on Oct 9, 2026

    

    dreampiggy

    on Oct 9, 2026

    Last edited by dreampiggy

    More actions

    The same issue. I have to use another Codex session to remove and filter all the image in the broken session, then resume the session and all issue fixed.

    clean_codex_session_images.py
23. neohazuki commented on Oct 13, 2026

    

    neohazuki

    on Oct 13, 2026

    More actions

    Problema persiste. 88 dias sem resolucao