Proposal: Preserve a Configurable Tail During Context Compaction #34963

What variant of Codex are you using?

CLI

What feature would you like to see?

Preserve a configurable tail during context compaction

I would like to propose preserving the last N conversation items during context compaction.

This would be especially useful for models that perform long execution loops involving testing, benchmarking, refactoring, and repeated validation.

A typical workflow may look like this:

code change
↓
run tests
↓
run benchmarks
↓
analyze results
↓
refactor
↓
run tests again

These operations can generate a large amount of context very quickly.

The problem appears when compaction removes or heavily summarizes the most recent execution state. After compaction, the model may still understand the general project, but it can lose important information such as:

the tests were already executed
the benchmark was already completed
a specific approach was already rejected
the next step was already decided

As a result, the model may start repeating the same operations again.

tests and benchmarks
↓
large amount of context
↓
context compaction
↓
recent execution state is lost
↓
tests and benchmarks are executed again
↓
another compaction
↓
the loop repeats

This can make complex tasks difficult or impossible to finish. It also wastes time, tokens, compute, and usage limits.

Proposed solution

Preserve a configurable raw tail during compaction.

[compacted older history] + [last N items preserved unchanged]

For example:

codex --compact-tail-items 5

Or in configuration:

[context]
compact_tail_items = 5

The preserved tail should ideally contain complete conversation and execution items, not only user messages.

This may include:

user messages
assistant messages
tool calls
tool results
test output
benchmark output

Otherwise, the system could preserve a message requesting a benchmark while removing the result showing that the benchmark was already completed.

A token-based option could also be useful:

[context]
compact_tail_items = 5
compact_tail_tokens = 12000

The system could preserve either the configured number of items, the configured token budget, or use a hybrid strategy.

The main goal is to preserve the active execution state, not only general knowledge about the task.

Optional improvement: asynchronous prefix compaction

Compaction could also begin slightly before the context limit is reached.

For example, it could start when approximately 5 percent of the context window remains, or earlier depending on the expected tail size.

Instead of blocking model execution:

model work
↓
context limit reached
↓
model pauses
↓
compaction runs
↓
model resumes

The process could work concurrently:

model work ─────────────────────────────────────────────►
└─ compact older prefix in background

The compactor would operate only on a stable snapshot of the older conversation prefix.

|------------ stable prefix ------------|---- live tail ----►
background compaction model continues

The model could continue generating new messages, tool calls, test results, and benchmark results inside the live tail.

When background compaction finishes, the old prefix could be replaced atomically:

before:

[old prefix] + [live tail]

after:

[compacted prefix] + [same live tail]

This is similar to techniques used in databases and concurrent systems.

The compactor works on a snapshot of stable data, while new changes continue to be appended separately. When the background operation finishes, the compacted snapshot replaces the old prefix without modifying the newer data.

Conceptually:

1. Create a snapshot at position T
2. Compact everything before T
3. Continue appending new items after T
4. Replace the old prefix with the compacted prefix
5. Keep everything created after T unchanged

This resembles multiversion concurrency control and snapshot-based processing in databases.

conversation version T:

[prefix being compacted] | [tail]

conversation continues:

[prefix being compacted] | [tail + new messages + tool results]

compaction completes:

[compacted prefix] | [tail + new messages + tool results]

The compactor should never process the moving end of the conversation. It should only process an immutable prefix.

|------ immutable prefix ------|------ mutable live tail ------►
safe to compact model keeps working

Even if the background summary does not include the latest three to five items, that is not a problem because those items remain available unchanged in the preserved tail.

Why this matters

Context compaction should preserve continuity of execution, not only continuity of information.

A summary may correctly preserve the architecture of a project while losing the current process state.

For example:

what has already been tested
which benchmark has already run
which solution has failed
which files were just changed
what the next exact action should be

This information behaves like working memory. It is local, recent, and essential for maintaining forward progress.

Without it, the model can remember the destination while forgetting the last few steps of the path.

Suggested implementation stages

The feature could be introduced in two stages.

Stage 1

Preserve a configurable raw tail during compaction.

[compacted history] + [unchanged recent items]

This would likely solve most repetition loops and should be relatively simple to implement.

Stage 2

Add speculative or asynchronous compaction of a stable conversation prefix.

model execution ───────────────────────────────────────►
└─ background prefix compaction

This could reduce or eliminate pauses caused by compaction while preserving the latest execution state.

The first stage addresses correctness and task continuity.

The second stage improves latency and allows the model to continue working while older context is being compacted.

Additional information

No response