Codex-generated test harness leaked unbounded Docker/Xcode resources without post-run verification #33707

What version of Codex was running?

The harness was generated during a codex-tui session on the 0.144.0 line. The current installed binary is codex-cli 0.144.5.

Which model was being used?

gpt-5.6-sol, reasoning effort ultra.

What platform was the computer using?

macOS 26.4 (25E246), arm64, with Docker Desktop and Xcode/iOS Simulator.

What issue occurred?

Codex designed and committed an iOS E2E/fault-test harness that created unbounded external resources and then repeatedly ran it without verifying resource cleanup.

The generated harness used:

• a unique Docker Compose project name for every fault scenario/run
• docker compose down --remove-orphans, but no removal of project-local images
• a new DerivedData path and .xcresult bundle for each run
• no retention policy for DerivedData or result bundles
• simulators that could be booted for a run but were not shut down afterward
• shared/global BuildKit cache with no run ownership or bound
• traps for the immediate shell, but no stale-run reaper for crashes/SIGKILL
• no post-run assertion that Docker/Xcode/simulator resource usage returned to baseline

This was not primarily an interrupt/cancellation failure. The resources accumulated after successful runs because the generated harness itself had incomplete lifecycle semantics.

After many E2E runs:

• the host had about 191 MB free
• Docker Desktop's physical Docker.raw usage was about 227 GB
• Docker Desktop crashed with no space left on device
• Xcode showed many retained test/run entries
• DerivedData, result bundles, project-local images, BuildKit cache, and simulator state had accumulated

After safely pruning only unused Docker data and repairing the harness (without removing Docker volumes), the host recovered more than 200 GB and Docker physical usage dropped to roughly 51 GB.

Reproduction pattern

This is a model/agent behavior reproduction rather than a deterministic CLI command:

1. Ask Codex to build a comprehensive iOS E2E matrix with Docker Compose-based transport-fault services and Xcode result bundles.
2. Let Codex design the runner and execute it repeatedly across multiple scenarios/devices.
3. Inspect the generated lifecycle code.
4. Run the suite repeatedly and compare:
   • Docker images, containers, builders, and build cache
   • DerivedData and .xcresult directories
   • booted simulators
   • free host disk
5. Observe that immediate containers are stopped, but run-owned images, cache, Xcode artifacts, and simulator state remain and grow without a bound.

Representative generated patterns were equivalent to:

compose_project="...-${run_id}-${label}-$$"
derived_data=".../${run_id}/${label}"
result_bundle=".../${run_id}/${label}/result.xcresult"

docker compose --project-name "$compose_project" up -d --build
# ...
docker compose --project-name "$compose_project" down --remove-orphans

Stopping the Compose project did not remove the uniquely named project image, and neither artifact path had retention.

What was expected?

When Codex generates and executes test infrastructure, it should treat every external resource it creates as owned state and verify a bounded lifecycle:

• cleanup on success, failure, and catchable signals
• explicit ownership markers
• stale-run recovery for crashes or SIGKILL
• bounded artifact retention
• cleanup of run-owned images/builders/cache without touching user volumes
• simulator shutdown when the runner booted it
• a post-run resource-delta check before reporting the task complete

The agent should not need to globally prune Docker or delete arbitrary Xcode data. It should generate scoped cleanup for resources it owns.

Why this appears to be an agent-quality issue

The CLI executed the generated scripts as written. It cannot safely infer that an arbitrary Docker image, simulator, or Xcode artifact is disposable. The defect was that Codex generated infrastructure with incomplete ownership and cleanup semantics, repeatedly ran it, and declared the work complete without checking the machine-level resource delta.

This therefore appears to be a model/agent planning and verification gap rather than primarily a shell process-tree bug.

Remediation applied locally

The repaired harness now includes:

• active-run markers and stale-run reaping
• run-owned DerivedData cleanup
• bounded result retention
• simulator shutdown only when the runner booted it
• Compose cleanup with run-owned local image removal
• a run-owned Buildx builder with a pinned BuildKit version and memory limit
• removal of the run-owned builder/state after the run
• explicit protection of Docker volumes

A useful regression/evaluation would ask the model to generate a multi-run Docker/Xcode test harness, run it several times, and assert that all created external resources are either removed or retained under a documented bound.