Design proposal: tier the dispatch verdict so unmodeled shapes degrade the check, not the turn #761

Short description

Tier the dispatch verdict so an unmodeled host shape degrades the check, not the turn: keep sending the same bytes (no cache change), count the divergence, and pin the session to admit-unverified after its first mismatch — then shrink the comparison from rows to tool-call windows so the largest divergence class stops existing at all.

What problem does this solve?

The recurrence, measured

Since the row-by-row comparison landed in 0.49.1, ten of the twenty 0.49 releases contain a fix for a turn refused or a request corrupted on this path:

Plus ten filed issues in the same window (#663, #667, #673, #686, #692, #698, #714, #723, #731, #751). #686 alone took 33 commits, and the fence family grew from 5 files / 102 KB at 0.49.1 to 14 files / 207 KB at 0.49.19 without ever removing a branch. 0.49.16 exists to undo a regression from 0.49.15.

That is not ten coincidences. It is one design assumption with a bounded return.

Why it recurs

1. The predicate is equality with a re-implementation of the host's encoder. piPreparedStructure ports transformMessages for 14 APIs × 2 hosts × 3 lanes, pinned to five upstream versions (18.2.6 … 18.12.5). Each fix adds one more ported behavior (e.g. 45c0bcff4 ports takeRealToolResult / flushPendingToolCalls). The set of histories where the port is exactly right is unbounded, so the residual is unbounded, and the host ships independently.
2. The check runs where it cannot act. At before_provider_request the payload is already built; refuse() notifies and calls ctx.abort(). No repair, no partial outcome.
3. The check is blind to the harm it exists for. Replacing every tool result with [dropped N] (All tool results come back as dropped n #731) keeps roles, counts and ids intact and passes; a well-formed request whose rows the host reordered (0.49.16: false 'comparison=role' refusal — the host pulls a tool result forward across an interrupting message, the prepared projection does not model the move #751) fails. The check is precise about shape and silent about meaning.
4. A fixed bug returns because the rule has many copies. "Where does this row land" is implemented per lane, per host, per phase (initial / chained / retry / rebuild), and new phases keep arriving (positioned system rows, WebSocket continuations).
5. The failure has no memory. 0.49.8: false 'comparison=content' refusal on cross-model assistant history (Responses lane) — 0.49.0 unaffected #714 and 0.49.16: false 'comparison=role' refusal — the host pulls a tool result forward across an interrupting message, the prepared projection does not model the move #751 reproduce at a stable row index on every retry. One unmodeled shape denies every subsequent turn in the session until a patch ships. The gate remembers what it sent (dispatchFields, preparedStructure) but never that it was wrong, so nothing self-limits — and that is what turns a modelling error into a release.

Proposed solution

1. Split the verdict into two questions

type PiDispatchVerdict =
  | { kind: "admit" }
  | { kind: "admit-unverified"; soft: PiSoftDivergence[] }  // log, count, send
  | { kind: "unsafe"; hard: PiHardViolation };              // repair, else degrade

Hard — the provider will actually reject this, or we lost our own content:

Soft — the host is entitled to do this, and the provider does not validate it:
row order and moved rows (the takeRealToolResult forward pull) · extra host rows (custom notices, reminders, compaction carriers) · _dupN id renames · replies from other models, phase, thinking demotion, grouping · cache_control stripping · image trimming · timestamp differences · leading system/developer placement.

Every item on the soft list has either caused a fix in the table above or is special-cased inside the projection today. None of them is a request the provider rejects; they are differences between our prediction and the host's encoder.

2. Failure memory — the smallest change that breaks the loop

The state we want already exists: structuralSkipReason makes piPreparedStructureMismatch return undefined, and the gate logs once and proceeds. Today it only engages when the plan is unmodelable at prepare time.

Extend it to observed failure: the first structural mismatch in a session admits, records a shape key (api, model, row count, first divergent role pair), logs once, and pins the session to skip-structure mode. Ownership and pairing stay enforced. Reset the pin on a model/API change or a new operation.

Result: a permanent, deterministic outage becomes one odd turn plus one log line, and it costs nothing in bytes.

3. Compare windows, not rows

Every failure above lives on the order axis. Compare tool-call windows — the set of calls in an assistant turn and the results that answer them — instead of positional rows. Order inside the history stops being observable, so the forward pull, grouping and reordering classes disappear by representation rather than by casework, and projectMissingToolResults (≈400 of the projection's 1,533 lines, almost all of it predicting where rows land) shrinks to a membership check. The comparison becomes set equality over windows plus the hard invariants.

4. Hard invariants where the array is still ours

Run the hard list on the array the context hook is about to return — the only point where a violation is still fixable. Repair once (restore or drop the offending row, rebuild from history); if it still fails, return the host's array for this turn (existing pass-through path) and mark the operation unmanaged. Refuse only if nothing legal can be produced.

5. Receipt by content, not position

piManagedSignature takes a marker (or a 128-char prefix) from one of the first 8 messages, and piHasManagedSignature scans 8 messages × 32 blocks × 4096 chars — fewer rows than a normal OMP turn prepends. Replace it with the hashes of the blocks we injected (lkgContentFields already provides a stable walk), matched by membership with a length prefilter. On a previous_response_id replay that re-sends no history, absence is expected and the chain anchor authorizes the pass.

6. Make the corpus automatic (adjacent, separate issue)

lkg_slots already stores the exact failing pass, and MAGIC_CONTEXT_PI_SERVED_BODY_CAPTURE already captures served bodies. Wiring capture → oracle fixture → CI turns each future divergence into a failing build instead of a user report, and issue-751-host-oracle.test.ts is the harness to grow.

Cache safety

The managed prefix is a byte-stable artifact here (exactReusablePrefix, replay fixtures, cacheWrite in the accounting), so the proposal has to be judged by what each tier does to the prefix:

The second row is the whole point: the classes that produced 0.49.2–0.49.5, #714 and #751 are soft, and admitting them sends exactly the bytes we send today. There is no cache cost to the part that removes most of the refusals.

The trap is an oscillating fallback — alternating managed and unmanaged prefixes makes every turn a miss, which is worse than either policy. That is why the session pins to one prefix and stays there, and why failure memory matters more than the fallback.

Lane note: these refusals are mostly Responses/Codex, where caching is implicit (longest-common-prefix match), so a tail change costs only the tail. Explicit-breakpoint lanes (Anthropic: writes ≈1.25× base, reads ≈0.1× at the 5-minute TTL) are where a full prefix change is expensive — and are not where these refusals come from.

What changes

• The gate gains a third verdict and a per-session shape pin; mismatch no longer aborts.
• The comparison unit becomes windows + hard invariants (stage 2), so the projection is no longer load-bearing for admission.
• The receipt becomes a content-hash set (stage 3); the 8-message/4096-char window goes away.
• Diagnostics: comparison=… strings survive as the soft vocabulary; doctor --issue gains divergence/degraded counters.
• Test contracts: wire-fence and structure tests move from admission assertions to classification assertions. Oracle fixtures are kept as corpus.

What does not change

• Bytes sent for admitted turns, including admit-unverified — prompt-cache stability and the system-delivery goldens hold.
• The planning side: tags, summaries, compartments, historian, dreamer, LKG content model, budget/force bands.
• The previous_response_id chain anchor semantics — it becomes the authority for replay, not a competitor to a text marker.
• Provider-facing request shape: no new fields, headers or metadata.
• No new user-facing configuration. (Stage 1 can sit behind an internal flag for one release, removable after.)
• OpenCode's existing path in stage 1, unless the invariant is placed there deliberately (see Harness coverage).
• Any of the existing oracle fixtures.

Why now, and in what order

The host moved 18.11.3 → 18.12.5 in days; any strategy whose period is longer than the host's change interval loses on structure regardless of effort. The real host oracle now exists (c8c0988eb, issue-751-host-oracle.test.ts), which makes both the corpus and the window comparison cheaper than they would have been two weeks ago.

Ordered by cache cost and diff size:

1. Failure memory + tiered verdict — patch-sized, no byte changes. Stops the P0 class.
2. Window comparison — minor-sized (0.50.x; it changes the diagnostic vocabulary). Shrinks the model structurally.
3. Hard invariants at prepare time — tail-only repair; the first tier that costs cache, and only when it acts.
4. Content receipt — removes the positional heuristic.
5. Corpus automation — the release gate that keeps all of the above from decaying.

Neither stage should land during a host-version bump window; stage 1 is the one to ship first because it is small, reversible, and testable against stored payloads.

Risks, cons, and how they are contained

• A shape we did not verify gets sent. True, and already true of the paths that reach structuralSkipReason. Contained by the hard invariants, which cover the things the provider rejects, and by ownership, which covers "is our content there". It does not cover meaning — that is a separate fix, and this proposal does not claim it.
• Regressions get masked. Mitigated by counting and surfacing every soft divergence in doctor --issue, and by failing CI on new divergence classes once the corpus is wired.
• The pin goes stale if the history changes shape later (model switch). Reset on api/model change and on a new operation.
• Behaviour change for users: a turn that would have been refused now proceeds. That is the intended trade — the refused turn was usually a legal request.
• Test churn in the most sensitive module. Real cost for a small team; I am offering to do the mechanical part (fixture reclassification) so review focuses on the policy.
• It does not fix everything. All tool results come back as dropped n #731-class corruption is a plan/artifact consistency problem; it needs edit obligations, and is called out as a follow-up rather than folded in here.

Expected ROI

• Availability: the structural-mismatch class (6 of the 10 releases above) stops being user-visible. Expected effect: at most one unaffected turn and one log line per (session, shape), instead of a dead turn on every retry.
• Cost: zero cache delta for the tiers that carry the win; the first cache-costing tier is rare by construction. Measure with cacheWrite per turn over a week to confirm — my prediction is no movement.
• Engineering: each host bump becomes a metric rather than an incident; the fix-latency requirement drops from hours to days, which is what makes the planned work possible.
• Code: stage 2 retires the adjudicating branch, not the predictor (≈400–500 lines of comparison plus three receipt fields eventually); the projection survives as diagnostics.
• Not in scope: semantic corruption, cancellation/abort races, and the OpenCode lanes beyond an explicit decision.

Acceptance criteria

1. Replaying stored payloads from lkg_slots of past failing sessions: every request that refused with comparison=role|content|count and has no hard violation is admitted.
2. One synthetic fixture per hard violation is repaired or degraded as specified; none aborts the turn.
3. Bytes sent for admitted turns are unchanged, including admit-unverified (system-delivery goldens do not move).
4. A session that hits one unmodeled shape shows exactly one divergence and then a stable prefix — no alternating prefixes; cacheWrite stays flat across following turns.
5. Divergences and degraded turns are counted and visible in doctor --issue with their class.
6. The gate runs on Pi as well as OMP (fenceEnabled is OMP-only today), or the reason it cannot is documented.

Falsifiable check for the premise: if most historical comparison= refusals contain a hard violation, this proposal is wrong and the edit pipeline is the priority. That is measurable from the LKG corpus before any code changes, and I am happy to run it first.

Alternatives considered

• Warn only, keep the comparison. Cheapest, but drops ownership on replay and leaves the check load-bearing for nothing. The tiered verdict keeps the useful half.
• Model the host more accurately. What 0.49.2–0.49.19 have been doing. It lowers the rate, cannot lower the class, and taxes every host release.
• Turn the fence off. Loses ownership and stale-pass detection.
• Retry the same pass on mismatch. Reproduces deterministically at the same row, so it multiplies the abort.
• Never edit inside an open tool window. Would remove the class by construction, but it conflicts with mid-turn budget pressure (Force band never folds during a long turn of a prefix-bound thinking model (Opus 5.5): context stays at 85-99% until the user writes again #729). A product trade, not a bug fix; worth naming but not folded in here.

Harness coverage

Following CONTRIBUTING.md: the invariant module is host-independent, which is the argument for placing it once rather than per harness. Stage 1 is Pi/OMP (where the check lives). OpenCode 1 and 2 need either the same module in their message path or an explicit not-applicable with a reason; the Rust module is N/A for dispatch verification unless it owns any pairing rules. I will fill the PR template's harness table honestly rather than assume either way.

Additional context

• Prior reports that feed this: 0.49.8: false 'comparison=content' refusal on cross-model assistant history (Responses lane) — 0.49.0 unaffected #714 (cross-model content equality), [pi/0.49.12] NUL-corrupted served-ledger JSONL files permanently block a session with a false 'storage is busy' refusal (Windows/Bun) #723 (durable-state integrity), 0.49.16: false 'comparison=role' refusal — the host pulls a tool result forward across an interrupting message, the prepared projection does not model the move #751 (host forward pull), plus the standalone comparison=role reproducer against the shipped 0.49.16 bundle (4-message case, drives the shipped fence to the identical verdict).
• The design detail behind each claim — the release table, the six signatures, the shadow-growth numbers, the failure-memory argument — is in a longer analysis I can post on request; this issue is the actionable subset.
• I can implement stage 1 (verdict + pin) behind an internal flag with the LKG replay test, and stage 6 (corpus automation) in parallel, once the shape is agreed. Draft PR until the design-approved label.