Activity

1. aniket-yamle

   added

   appIssues related to the Codex desktop appIssues related to the Codex desktop app

   on May 20, 2026
2. github-actions

   added

   bugSomething isn't workingSomething isn't working

   performance

   subagentIssues involving subagents or multi-agent featuresIssues involving subagents or multi-agent features

   on May 20, 2026
3. github-actions commented on May 20, 2026

   

   github-actionsBot

   on May 20, 2026 – with GitHub Actions

   Contributor

   More actions

   Potential duplicates detected. Please review them and close your issue if it is a duplicate.

   • CLI: subagents memory leak #23260

   Powered by Codex Action
4. jshaofa-ui commented on May 21, 2026

   

   jshaofa-ui

   on May 21, 2026

   More actions

   Fix: Memory Leak When Running Multiple Agents — Crashed at 128GB on MacBook

   Issue References

   • Primary: openai/codex Memory leak when running multiple agents #23749 — "Memory leak when running multiple agents"
   • Duplicate: openai/codex CLI: subagents memory leak #23260 — "CLI: subagents memory leak"
   • Feedback ID: 019e4666-b55f-7482-ba26-9db616a3c078

   ---

   Executive Summary

   When running parallel subagents, the Codex CLI process accumulates unbounded memory because:

   1. Unbounded event channels are never drained — Each spawned subagent creates an async_channel::unbounded() event channel, but after the subagent completes, the channel is never consumed or closed, causing events to accumulate indefinitely in memory.
   2. Completed subagent sessions remain live — The ThreadManagerState.threads HashMap retains Arc<CodexThread> references to completed subagents, preventing garbage collection of their session state, event channels, and associated runtime allocations.
   3. The completion watcher holds references — The detached tokio::spawn task in maybe_start_completion_watcher retains Arc references to AgentControl and ThreadManagerState, creating a reference cycle that prevents cleanup.
   4. AgentRegistry metadata is never purged — Completed subagent entries in AgentRegistry.active_agents.agent_tree persist for the lifetime of the parent session.

   ---

   Root Cause Analysis

   Cause 1: Unbounded Event Channel Never Drained

   File: codex-rs/core/src/session/mod.rs

   // Line 480-481: Unbounded event channel is created per session
   let (tx_sub, rx_sub) = async_channel::bounded(SUBMISSION_CHANNEL_CAPACITY);
   let (tx_event, rx_event) = async_channel::unbounded();  // <-- UNBOUNDED

   The tx_event sender is cloned into the Session and used throughout the session lifecycle to emit events (turn completions, tool calls, status changes, etc.). The rx_event receiver is exposed via Codex::next_event() (line 733), but only the first event (SessionConfigured) is consumed during thread finalization:

   // codex-rs/core/src/thread_manager.rs, line 1239-1254
   async fn finalize_thread_spawn(
       &self,
       codex: Codex,
       thread_id: ThreadId,
       session_source: SessionSource,
   ) -> CodexResult<NewThread> {
       let event = codex.next_event().await?;  // <-- ONLY consumes SessionConfigured
       let session_configured = match event {
           Event {
               id,
               msg: EventMsg::SessionConfigured(session_configured),
           } if id == INITIAL_SUBMIT_ID => session_configured,
           _ => {
               return Err(CodexErr::SessionConfiguredNotFirstEvent);
           }
       };
       // ... rest of registration
   }

   After the initial SessionConfigured event, no consumer ever drains the rx_event channel for subagents. The parent thread's Codex struct holds the rx_event receiver, but the parent never calls next_event() on child threads. This means:

   • Every Event sent via tx_event.send() during the subagent's lifetime is buffered in the unbounded channel
   • Events include: TurnComplete, ToolCall, ToolCallOutput, AgentMessage, TokenUsage, etc.
   • For long-running or high-activity subagents, this can accumulate thousands of events
   • The channel buffer grows without bound until the process exhausts memory

   Cause 2: Completed Child Sessions Not Released

   File: codex-rs/core/src/thread_manager.rs

   // Line 199-217: ThreadManagerState holds all threads in an Arc<RwLock<HashMap>>
   pub(crate) struct ThreadManagerState {
       threads: Arc<RwLock<HashMap<ThreadId, Arc<CodexThread>>>>,  // <-- NEVER cleaned up
       // ...
   }

   When a subagent completes its work, the ThreadManagerState.threads HashMap is never automatically cleaned up. The only removal paths are:

   1. Explicit close_agent() call — agent/control.rs:785-796 — requires the model to explicitly invoke close_agent tool
   2. shutdown_live_agent() — agent/control.rs:763-781 — only called during explicit shutdown
   3. handle_thread_request_result() on InternalAgentDied — agent/control.rs:754-758 — only on error

   // agent/control.rs:763-781 — explicit close path only
   pub(crate) async fn shutdown_live_agent(&self, agent_id: ThreadId) -> CodexResult<String> {
       let state = self.upgrade()?;
       let result = if let Ok(thread) = state.get_thread(agent_id).await {
           thread.codex.session.ensure_rollout_materialized().await;
           thread.codex.session.flush_rollout().await?;
           // ... send shutdown op
       };
       let _ = state.remove_thread(&agent_id).await;  // <-- Only removed here
       self.state.release_spawned_thread(agent_id);
       result
   }

   The completion watcher (maybe_start_completion_watcher) notifies the parent but does not clean up the child's resources:

   // agent/control.rs:991-1064
   fn maybe_start_completion_watcher(
       &self,
       child_thread_id: ThreadId,
       session_source: Option<SessionSource>,
       child_reference: String,
       child_agent_path: Option<AgentPath>,
   ) {
       // ...
       tokio::spawn(async move {
           // Watches status until terminal, then notifies parent
           // BUT NEVER calls shutdown_live_agent or remove_thread
           if child_agent_path.is_some() && /* MultiAgentV2 */ {
               // Sends InterAgentCommunication to parent
           } else {
               parent_thread.inject_user_message_without_turn(message).await;
           }
           // <-- Task ends, but child thread remains in threads HashMap
       });
   }

   Cause 3: Reference Cycle via Completion Watcher

   File: codex-rs/core/src/agent/control.rs

   The completion watcher spawns a detached tokio::task that holds a clone of AgentControl:

   // Line 1004-1063
   let control = self.clone();  // <-- Clones AgentControl (Arc<AgentRegistry> + Weak<ThreadManagerState>)
   tokio::spawn(async move {
       // The task holds `control` which keeps `AgentRegistry` alive
       // Even after the task completes, the Arc reference is dropped
       // but the child thread remains in ThreadManagerState.threads
   });

   While AgentControl uses Weak<ThreadManagerState> to avoid a direct reference cycle, the detached task still:

   • Keeps Arc<AgentRegistry> alive during its lifetime
   • The AgentRegistry holds active_agents.agent_tree with metadata for all spawned threads
   • The used_agent_nicknames HashSet grows without bound as new agents are spawned

   Cause 4: Forked Rollout Items Carry Full Parent History

   File: codex-rs/core/src/agent/control.rs

   When using SpawnAgentForkMode::FullHistory, the child receives a copy of the parent's entire rollout history:

   // Line 360-496: spawn_forked_thread
   let parent_history = state.read_stored_thread(...).await?.history.ok_or_else(...)?;
   let mut forked_rollout_items = parent_history.items;  // <-- Full copy
   // Items are filtered but the copy already happened

   For large sessions with many turns, each forked subagent carries a full copy of the parent's conversation history in its InitialHistory::Forked(...) variant.

   ---

   Proposed Fix

   Fix 1: Auto-Drain Event Channel on Subagent Completion

   Add an event drain task that runs after the subagent's submission loop terminates, consuming and discarding any remaining events.

   File: codex-rs/core/src/thread_manager.rs

   // In finalize_thread_spawn, after registering the thread, spawn a drain task
   // for subagent threads that will run when the submission channel closes.
   
   async fn finalize_thread_spawn(
       &self,
       codex: Codex,
       thread_id: ThreadId,
       session_source: SessionSource,
   ) -> CodexResult<NewThread> {
       let event = codex.next_event().await?;
       let session_configured = match event {
           Event {
               id,
               msg: EventMsg::SessionConfigured(session_configured),
           } if id == INITIAL_SUBMIT_ID => session_configured,
           _ => {
               return Err(CodexErr::SessionConfiguredNotFirstEvent);
           }
       };
   
       {
           let mut threads = self.threads.write().await;
           if let std::collections::hash_map::Entry::Vacant(e) = threads.entry(thread_id) {
               let thread = Arc::new(CodexThread::new(
                   codex.clone(),
                   session_configured.clone(),
                   session_configured.rollout_path.clone(),
                   session_source.clone(),
               ));
               e.insert(thread.clone());
   
               // NEW: Spawn event drain task for subagent threads
               if session_source.is_non_root_agent() {
                   let rx_event = codex.rx_event.clone();
                   let thread_id_for_drain = thread_id;
                   tokio::spawn(async move {
                       // Drain all remaining events after the submission loop closes.
                       // This prevents the unbounded channel from accumulating events.
                       while let Ok(_event) = rx_event.recv().await {
                           // Discard events — they've been persisted to rollout
                       }
                       tracing::debug!(
                           thread_id = %thread_id_for_drain,
                           "subagent event channel drained"
                       );
                   });
               }
   
               return Ok(NewThread {
                   thread_id,
                   thread,
                   session_configured,
               });
           }
       }
   
       if let Err(err) = codex.shutdown_and_wait().await {
           warn!("failed to shut down duplicate thread {thread_id}: {err}");
       }
       Err(CodexErr::InvalidRequest(format!(
           "thread {thread_id} is already running"
       )))
   }

   Fix 2: Auto-Cleanup Completed Subagents

   Modify the completion watcher to automatically clean up completed subagents by calling shutdown_live_agent after notification.

   File: codex-rs/core/src/agent/control.rs

   fn maybe_start_completion_watcher(
       &self,
       child_thread_id: ThreadId,
       session_source: Option<SessionSource>,
       child_reference: String,
       child_agent_path: Option<AgentPath>,
   ) {
       let Some(SessionSource::SubAgent(SubAgentSource::ThreadSpawn {
           parent_thread_id, ..
       })) = session_source
       else {
           return;
       };
       let control = self.clone();
       tokio::spawn(async move {
           let status = match control.subscribe_status(child_thread_id).await {
               Ok(mut status_rx) => {
                   let mut status = status_rx.borrow().clone();
                   while !is_final(&status) {
                       if status_rx.changed().await.is_err() {
                           status = control.get_status(child_thread_id).await;
                           break;
                       }
                       status = status_rx.borrow().clone();
                   }
                   status
               }
               Err(_) => control.get_status(child_thread_id).await,
           };
           if !is_final(&status) {
               return;
           }
   
           let Ok(state) = control.upgrade() else {
               return;
           };
           let child_thread = state.get_thread(child_thread_id).await.ok();
           let message = format_subagent_notification_message(child_reference.as_str(), &status);
   
           // Notify parent first
           if child_agent_path.is_some()
               && child_thread
                   .as_ref()
                   .map(|thread| thread.enabled(Feature::MultiAgentV2))
                   .unwrap_or(true)
           {
               let Some(child_agent_path) = child_agent_path.clone() else {
                   return;
               };
               let Some(parent_agent_path) = child_agent_path
                   .as_str()
                   .rsplit_once('/')
                   .and_then(|(parent, _)| AgentPath::try_from(parent).ok())
               else {
                   return;
               };
               let communication = InterAgentCommunication::new(
                   child_agent_path,
                   parent_agent_path,
                   Vec::new(),
                   message,
                   /*trigger_turn*/ false,
               );
               let _ = control
                   .send_inter_agent_communication(parent_thread_id, communication)
                   .await;
           } else if let Ok(parent_thread) = state.get_thread(parent_thread_id).await {
               parent_thread
                   .inject_user_message_without_turn(message)
                   .await;
           }
   
           // NEW: Auto-cleanup completed subagent
           // Flush rollout and remove from threads HashMap to free memory
           if let Some(thread) = state.get_thread(child_thread_id).await.ok() {
               thread.ensure_rollout_materialized().await;
               let _ = thread.flush_rollout().await;
               // Wait for the session loop to terminate
               thread.wait_until_terminated().await;
           }
           let _ = state.remove_thread(&child_thread_id).await;
           control.state.release_spawned_thread(child_thread_id);
   
           tracing::debug!(
               child_thread_id = %child_thread_id,
               parent_thread_id = %parent_thread_id,
               "auto-cleaned up completed subagent"
           );
       });
   }

   Fix 3: Bounded Event Channel for Subagents

   Replace the unbounded event channel with a bounded channel for subagent sessions, with a size that balances memory usage against the risk of blocking the submission loop.

   File: codex-rs/core/src/session/mod.rs

   // Add constant for subagent event channel capacity
   const SUBAGENT_EVENT_CHANNEL_CAPACITY: usize = 256;
   
   // In Codex::spawn_internal:
   let (tx_sub, rx_sub) = async_channel::bounded(SUBMISSION_CHANNEL_CAPACITY);
   
   // Use bounded channel for subagents to prevent unbounded memory growth
   let (tx_event, rx_event) = if session_source.is_non_root_agent() {
       async_channel::bounded(SUBAGENT_EVENT_CHANNEL_CAPACITY)
   } else {
       async_channel::unbounded()
   };

   Note: With a bounded channel, the send_event calls must handle the SendError case where the channel is full. Since the drain task (Fix 1) will consume events, this should only happen under extreme load. Consider adding a warning log when events are dropped.

   Fix 4: Truncate Forked History for Subagents

   Add a configuration option to limit the amount of parent history copied to forked subagents.

   File: codex-rs/core/src/agent/control.rs

   // In spawn_forked_thread, add history size limit
   const MAX_FORK_HISTORY_ITEMS: usize = 1000;
   
   let mut forked_rollout_items = parent_history.items;
   
   // NEW: Limit forked history size to prevent memory blowup
   if forked_rollout_items.len() > MAX_FORK_HISTORY_ITEMS {
       let skip = forked_rollout_items.len() - MAX_FORK_HISTORY_ITEMS;
       forked_rollout_items.drain(..skip);
       tracing::warn!(
           parent_thread_id = %parent_thread_id,
           original_items = parent_history.items.len(),
           truncated_items = forked_rollout_items.len(),
           "truncated forked history to prevent memory blowup"
       );
   }

   Fix 5: Periodic AgentRegistry Cleanup

   Add a periodic cleanup mechanism for the AgentRegistry to remove stale metadata entries.

   File: codex-rs/core/src/agent/registry.rs

   impl AgentRegistry {
       /// Remove metadata entries for threads that no longer exist in the thread manager.
       /// This should be called periodically to prevent metadata accumulation.
       pub(crate) async fn cleanup_stale_entries(
           &self,
           active_thread_ids: &HashSet<ThreadId>,
       ) {
           let mut active_agents = self
               .active_agents
               .lock()
               .unwrap_or_else(std::sync::PoisonError::into_inner);
   
           let stale_keys: Vec<String> = active_agents
               .agent_tree
               .iter()
               .filter_map(|(key, metadata)| {
                   if let Some(thread_id) = metadata.agent_id {
                       if !active_thread_ids.contains(&thread_id) && !AgentPath::from_str(key).map_or(false, AgentPath::is_root) {
                           return Some(key.clone());
                       }
                   }
                   None
               })
               .collect();
   
           for key in &stale_keys {
               if let Some(metadata) = active_agents.agent_tree.remove(key) {
                   if metadata.agent_path.as_ref().is_some_and(AgentPath::is_root) {
                       continue;
                   }
                   active_agents.used_agent_nicknames.remove(
                       metadata.agent_nickname.as_ref().unwrap_or(&String::new())
                   );
               }
           }
   
           if !stale_keys.is_empty() {
               tracing::debug!(
                   removed_count = stale_keys.len(),
                   "cleaned up stale agent registry entries"
               );
           }
       }
   }

   ---

   Implementation Plan

   Phase 1: Critical Fixes (High Impact, Low Risk)

   1. Fix 2 — Auto-Cleanup Completed Subagents

      • File: codex-rs/core/src/agent/control.rs
      • Lines: 991-1064 (maybe_start_completion_watcher)
      • Risk: Low — adds cleanup after existing notification logic
      • Testing: Verify subagent memory is released after completion

   2. Fix 1 — Auto-Drain Event Channel

      • File: codex-rs/core/src/thread_manager.rs
      • Lines: 1239-1280 (finalize_thread_spawn)
      • Risk: Low — adds a background drain task that runs after channel close
      • Testing: Verify event channel doesn't grow unbounded

   Phase 2: Defensive Fixes (Medium Impact, Medium Risk)

   3. Fix 3 — Bounded Event Channel for Subagents

      • File: codex-rs/core/src/session/mod.rs
      • Lines: 480-481 (Codex::spawn_internal)
      • Risk: Medium — bounded channels can block if not drained fast enough
      • Testing: Verify no event loss under high load

   4. Fix 4 — Truncate Forked History

      • File: codex-rs/core/src/agent/control.rs
      • Lines: 410-415 (spawn_forked_thread)
      • Risk: Low — only affects full-history fork mode
      • Testing: Verify forked subagents still have sufficient context

   Phase 3: Monitoring & Prevention (Low Impact, Low Risk)

   5. Fix 5 — Periodic AgentRegistry Cleanup
      • File: codex-rs/core/src/agent/registry.rs
      • Lines: 79-167 (AgentRegistry impl)
      • Risk: Low — cleanup only removes entries for already-removed threads
      • Testing: Verify cleanup doesn't remove active entries

   ---

   Testing Plan

   Memory Profiling Tests

   # Build with profiling support
   cd codex-rs
   cargo build --release
   
   # Run with memory profiling
   # 1. Baseline: Single agent, no subagents
   valgrind --tool=massif --stacks=yes ./target/release/codex \
       --prompt "Write a simple sorting function" \
       --model o4-mini 2> massif.out
   
   # 2. Test: Multiple parallel subagents
   # Create a prompt that triggers many spawn_agent calls
   cat > /tmp/test_prompt.txt << 'EOF'
   For each of these files, create a subagent to analyze it:
   - src/main.rs
   - src/lib.rs
   - src/config.rs
   - src/utils.rs
   - src/error.rs
   - src/types.rs
   - src/parser.rs
   - src/lexer.rs
   - src/ast.rs
   - src/codegen.rs
   EOF
   
   # Run with subagent stress test
   time ./target/release/codex \
       --prompt-file /tmp/test_prompt.txt \
       --model o4-mini
   
   # Monitor memory growth
   ps aux | grep codex  # Check RSS

   Automated Memory Test

   // codex-rs/core/src/agent/control_tests.rs
   
   #[tokio::test]
   async fn subagent_completion_releases_memory() {
       // Spawn multiple subagents and verify memory is released after completion
       let manager = ThreadManager::with_models_provider_for_tests(
           auth,
           provider,
       ).await;
   
       let initial_threads = manager.state.threads.read().await.len();
   
       // Spawn 10 subagents
       for i in 0..10 {
           let _ = manager.start_thread_with_options(StartThreadOptions {
               config: test_config(),
               initial_history: InitialHistory::New,
               session_source: Some(SessionSource::SubAgent(SubAgentSource::ThreadSpawn {
                   parent_thread_id: root_thread_id,
                   depth: 1,
                   agent_path: Some(AgentPath::try_from(&format!("agent-{i}")).unwrap()),
                   agent_nickname: None,
                   agent_role: None,
               })),
               // ...
           }).await;
       }
   
       // Wait for completion (simulated)
       tokio::time::sleep(Duration::from_secs(5)).await;
   
       // Verify threads were cleaned up
       let final_threads = manager.state.threads.read().await.len();
       assert_eq!(final_threads, initial_threads + 1); // Only root thread remains
   }
   
   #[tokio::test]
   async fn event_channel_drained_after_completion() {
       // Verify that the event channel is drained after subagent completion
       // This test checks that the drain task consumes all events
   }

   Integration Test Scenario

   # Simulate the reported issue scenario
   # 1. Start Codex with a large codebase
   # 2. Use a prompt that triggers parallel subagent spawning
   # 3. Monitor memory usage over time
   
   # Expected behavior after fix:
   # - Memory should stabilize after subagent completion
   # - RSS should not grow beyond ~500MB even with 100+ subagents
   # - Thread count should return to baseline after all subagents complete

   ---

   Edge Cases & Considerations

   1. Parent Thread Exits Before Child Completes

   If the parent thread shuts down before the child completes, the completion watcher's Weak<ThreadManagerState> upgrade will fail. The current code already handles this by returning early. With the fix, the cleanup code path must also handle this gracefully.

   2. Subagent Crashes (Non-Terminal Status)

   If a subagent crashes without reaching a terminal status, the completion watcher may hang indefinitely. Consider adding a timeout:

   let status = match tokio::time::timeout(
       Duration::from_secs(3600), // 1 hour max
       async {
           // ... existing status watching logic
       }
   ).await {
       Ok(status) => status,
       Err(_) => {
           tracing::warn!(
               child_thread_id = %child_thread_id,
               "subagent completion watcher timed out, cleaning up"
           );
           AgentStatus::Failed // Treat as terminal
       }
   };

   3. Nested Subagents (Depth > 1)

   Subagents that spawn their own subagents must also clean up their children. The auto-cleanup in maybe_start_completion_watcher handles this recursively because each subagent has its own watcher.

   4. Explicit close_agent Called Before Auto-Cleanup

   If the user/model explicitly calls close_agent before the completion watcher triggers, the thread is already removed from the HashMap. The auto-cleanup must handle ThreadNotFound gracefully:

   // Already handled: state.remove_thread returns Option, ignoring None
   let _ = state.remove_thread(&child_thread_id).await;

   5. Event Channel Drain Task Panics

   If the drain task panics, it should not affect the parent process. The tokio::spawn handles this by default (panics in spawned tasks don't propagate).

   6. Bounded Channel Blocking

   If the bounded event channel fills up faster than the drain task can consume, send_event calls will block. This could stall the submission loop. Mitigations:

   • Use a sufficiently large channel capacity (256 should be enough for most cases)
   • Add a timeout to send_event calls for subagents
   • Log warnings when events are dropped

   ---

   Prevention Strategies

   1. Add Memory Budget Configuration

   # config.toml
   [multi_agent]
   # Maximum number of concurrent subagents
   max_concurrent = 16
   # Maximum memory per subagent (bytes)
   memory_budget_per_agent = 256_000_000  # 256MB
   # Auto-close subagents after inactivity (seconds)
   auto_close_timeout = 300

   2. Add Telemetry for Memory Monitoring

   // In agent/control.rs, after cleanup:
   if let Some(metrics) = codex_otel::global() {
       let _ = metrics.gauge(
           "codex.multi_agent.active_threads",
           threads.read().await.len() as f64,
           &[],
       );
       let _ = metrics.gauge(
           "codex.multi_agent.registry_entries",
           active_agents.agent_tree.len() as f64,
           &[],
       );
   }

   3. Add Health Check Endpoint

   // Report memory usage and thread count for monitoring
   pub async fn health_check(&self) -> HealthStatus {
       let threads = self.state.threads.read().await;
       HealthStatus {
           active_threads: threads.len(),
           registry_entries: self.state.agent_registry.active_agents.lock().unwrap().agent_tree.len(),
           total_spawned: self.state.agent_registry.total_count.load(Ordering::Acquire),
       }
   }

   4. Document Best Practices

   Add documentation for users about:

   • Recommended max concurrent subagents based on available memory
   • When to use close_agent explicitly vs. relying on auto-cleanup
   • How to monitor memory usage during parallel agent execution

   ---

   Summary of Changes

   File	Lines	Change
   codex-rs/core/src/thread_manager.rs	1239-1280	Add event drain task for subagents
   codex-rs/core/src/agent/control.rs	991-1064	Auto-cleanup completed subagents in completion watcher
   codex-rs/core/src/agent/control.rs	410-415	Truncate forked history for memory safety
   codex-rs/core/src/session/mod.rs	480-481	Use bounded event channel for subagents
   codex-rs/core/src/agent/registry.rs	79-167	Add periodic cleanup for stale entries

   ---

   Expected Impact

   Based on the observations from issue #23260:

   Metric	Before Fix	After Fix
   RSS growth (23 MB → 315 MB+)	Unbounded	Bounded to ~50 MB per active subagent
   malloc allocation count (79k → 691k)	Unbounded	Stabilizes after cleanup
   Thread count (22 → 68)	Accumulates	Returns to baseline (~22)
   vmmap physical footprint	~363.7 MB+	~50-100 MB per active subagent
   Event channel buffer	Unbounded	0 (drained) or bounded (256)

   The fix should prevent the 128GB memory crash scenario by ensuring that completed subagents release all their resources promptly.
5. sigyulee commented on May 21, 2026

   

   sigyulee

   on May 21, 2026

   More actions

   Hello from #23260

   My understanding is that memory usage spikes because each subagent receives the full transcript from the parent, and MCPs are started for each subagent. Keeping the context and MCP catalog clean may help.
6. Taridoku commented on Jun 15, 2026

   

   Taridoku

   on Jun 15, 2026

   More actions

   Corroborating evidence: native app-server heap, attributed to streamed-response buffers across concurrent subagent sessions

   Reproduced on the VS Code extension and captured the memory composition while it ballooned.

   Environment: VS Code Codex extension openai.chatgpt 26.609.30741, macOS 26.3.1, Apple Silicon, 17 GB RAM, gpt-5.5, model_reasoning_effort = "xhigh", [features] multi_agent = true.

   Repro: asked Codex to perform a large code-audit task. Native multi-agent fanned out to 5 new concurrent session_loop sessions spawned within 18 seconds (from the app-server's own logs):

   11:21:00  new session_loop 019ecacc-806a...  op: UserInput
   11:21:06  new session_loop 019ecacc-9734...  op: UserInput
   11:21:06  new session_loop 019ecacc-9792...  op: UserInput
   11:21:12  new session_loop 019ecacc-af54...  op: UserInput
   11:21:18  new session_loop 019ecacc-c66e...  op: UserInput
   

   Memory went from 1.7 GB to 14 GB in ~2 minutes; only an explicit op: Interrupt released it. Left unattended it freezes the machine (hard restart required).

   Memory attribution (vmmap on the live native codex app-server process):

   Physical footprint:        14.2G
   Physical footprint (peak): 17.0G   (= total RAM)
   Writable regions: swapped_out 11.1G (68%)
   MALLOC_LARGE     9.4G  (81 regions)
   MALLOC_REALLOC   4.3G  (37 regions)   <- continually re-grown buffers
   

   The footprint is almost entirely heap. The MALLOC_REALLOC mass (buffers repeatedly doubled) points to unbounded accumulation, and the busiest log targets during the spike were codex_api::endpoint::responses_websocket and codex_api::sse::responses, i.e. streamed response bodies retained per concurrent session and not released until every session completes. sample showed hot frames in codex_core::tools::parallel::ToolCallRuntime and codex_app_server_transport::transport::serialize_outgoing_message.

   Net: N concurrent subagent sessions each retain their full streamed-response context in the parent app-server heap with no cap. On a 17 GB host, roughly 5 to 6 wide is enough to exhaust RAM and freeze the OS. A per-session buffer cap, or releasing completed-stream buffers before all siblings finish, would bound it.
7. Patchy

   added 2 commits that reference this issue on Aug 11, 2026

   tui: fix unbounded memory growth from background agent event buffers

   ...

   201b314

   tui: free event buffers for closed background agent threads

   ...

   97dccc8