Skip to content

bug(omp): first turn after a restart is refused — a cancelled tag-snapshot prewarm is treated as a hydration failure (0.49.22) #790

Closed

Description

@Zireael

Short description

On OMP, the first user turn after a restart is refused whenever the process also starts clone sessions:
stage=tag snapshot hydration … refused (Pi tag hydration: cancelled). Five startups in a row, 2026-10-25 → 2026-10-26.
The turn never reaches the model and the message is lost. A cancelled prewarm is being treated as a failed
hydration, even though no snapshot was ever read.

What happened

Plugin: @cortexkit/pi-magic-context 0.49.22. OMP 18.12.6, Responses lane (openai-codex-responses, gpt-6.1-sol).
Long session (01a1584d…, 761 messages), two clone sessions created at startup.

In a 27-hour log with 56 extension registrations, every startup that created two clone sessions refused its first
turn (5 of 5)
, and none of the other 51 registrations did. All five land 0.6–0.8 s after the plugin registers its
handlers, each preceded by exactly two clone-session migrations from the same source session:

08:07:23.938 [magic-context][pi] registered context handler (tagging + drops + nudges)
08:07:24.393 [magic-context][pi] clone-inheritance: migrated compartments=0 tags=0 notes=0 facts=0
             source=01a1584d-… dest=01a1773f-f8b2-… reason=startup
08:07:24.421 [magic-context][pi] clone-inheritance: migrated … dest=01a1773f-f8c4-… reason=startup
08:07:24.474 [magic-context][…-f8b2] system prompt hash initialized: 1f5e45d1… (len=41028)
08:07:24.489 [magic-context][…-f8c4] system prompt hash initialized: ffb4ccc6… (len=41069)
08:07:24.526 [magic-context][pi] turn refused Pi tag hydration: cancelled
PiTagHydrationError: Pi tag hydration: cancelled
    at abort (…/dist/index-5j6vwg4z.js:410:30)
    at <anonymous> (…:596:15)
08:07:24.526 [magic-context][pi] CONTEXT REFUSED "Magic Context could not safely prepare this turn; send your
             message again. stage=tag snapshot hydration elapsed=2ms recovery=no managed result;
             refused (Pi tag hydration: cancelled)"

The other four: 2026-10-25 06:04:02.842Z, 2026-10-25 18:54:59.014Z, 2026-10-26 06:34:25.601Z,
2026-10-26 09:14:30.778Z — same sequence, same two migrations, elapsed=2–30ms.

Expected: the first turn after a restart is served (possibly after a short hydration wait).
Observed: it is aborted, and sending it again works only because the second pass happens after the prewarms settle.

Why it happens

Three facts combine:

  1. Every started session prewarms. session_start ends with prewarmPiTagSnapshot(db, sessionId) (index.ts),
    so a restart with two clones starts three hydrations. session_switch prewarms the incoming session the same way.

  2. Hydration has one shared worker slot, and load() cancels across sessions (tag-snapshot-pi.ts):

    if (current) {
        if (current.sessionId === sessionId && !current.stale) return current.result;
        current.cancel();                       // cancels a *different* session's job
        await current.released;
        …
    }

    current.cancel() reaches abort() with no argument, i.e.
    new PiTagHydrationError("cancelled") (tag-snapshot-worker-client-pi.ts:402).
    cancelPiTagSnapshotSession() (session switch / shutdown / db.close) produces the identical error, so both
    paths are in play at startup; a diagnostic line in cancel() naming the session and caller would pin it down.

  3. A cancellation is not retried. ensurePiTagSnapshotReady() retries only stale:

    if (!(error instanceof PiTagHydrationError && error.code === "stale"))
        throw error;

    "cancelled" has no code, so it propagates out of the pass and the turn is refused.

A cancelled prewarm carries no evidence about the request: nothing was read, so there is nothing the fence could be
protecting against.

Proposed fix

  1. tag-snapshot-pi.ts load(): never cancel another session's job to start this one — await released and queue
    (the promise already exists), or keep one job per session with a small pool.
  2. ensurePiTagSnapshotReady(): retry cancelled exactly like stale when this pass did not cancel the job
    (pass a cancel token so a user abort still aborts immediately).
  3. If the retry budget is exhausted, serve what the pass has (awaited snapshot or raw/served messages) instead of
    refusing. A failed prewarm must not fail a turn.
  4. Skip prewarmPiTagSnapshot for clone destinations created reason=startup — the session_switch path already
    skips reason=fork for the same cache-noise reason.

A regression test would prewarm A, start B while A is still reading, and assert that a pass on A is served rather
than refused.

Environment

  • Plugin: @cortexkit/pi-magic-context 0.49.22 (master 0.49.23 has the same three code paths)
  • Harness: OMP (opencode-free-omp) 18.12.6, --harness pi, Bun
  • Platform: Windows x64
  • API: openai-codex-responses; session mixes tool calls and compaction markers; two clone sessions per startup
  • Log: %TEMP%\omp\magic-context\magic-context.log

Activity

  1. magic-alfonso commented on Oct 26, 2026

    @magic-alfonso

    Thanks for another precise report. The startup sequence from five restarts made the pattern clear.

    Confirmed. load() in tag-snapshot-pi.ts cancels a different session's hydration job to start its own, and ensurePiTagSnapshotReady() retries only stale. So a prewarm cancelled by the clone sessions starting up surfaces as a refused turn, even though nothing about the request was wrong.

    We're working on a fix: one session's hydration will no longer cancel another's, and a cancellation the pass didn't cause will be retried rather than refusing the turn. I'll post here when it ships.

  2. TheOrdinaryWow commented on Oct 26, 2026

    @TheOrdinaryWow

    Another trigger for the same load() cross-session cancel, in case it helps the regression test: parallel subagent spawns. No restart or clone sessions are needed.

    Environment: @cortexkit/pi-magic-context 0.49.22 (0.49.23 ships a byte-identical index-5j6vwg4z.js), omp 18.13.0 (also seen on 18.12.6). Subagents run in the parent's process, so they share the DB handle and its single hydration slot.

    When the parent's task call starts 4–5 children in the same ~25 ms window, the children cancel each other's hydration. Each cancelled child's first turn is refused, ctx.abort() runs, and the host records the turn as stopReason: "aborted", errorMessage: "Interrupted by user". The host's idle reminder then retries three times within ~100 ms. Those retries race the same siblings again, so a child often exhausts its reminders and ends as cancelled · (no output) after 4 req less than a second after spawning. From the parent's side it looks like the subagent interrupted itself.

    One batch (children started within 23 ms of each other):

    child Pi tag hydration: cancelled refusals outcome
    A 5 cancelled, 0 served turns
    B 2 (then the follow-on below) cancelled, 0 served turns
    C 4 cancelled; served after the parent messaged it
    D (started last) 0 completed

    Across 15 children in two parent sessions, all 40 Interrupted by user aborts are immediately preceded by a magic-context-turn-refused entry, and every child without one completed normally. A second batch of 5 parallel children reproduced it (4 of 5 refused, elapsed=0–102ms).

    The fix described above (no cross-session cancel, and a retry for cancellations the pass didn't cause) should cover this path too. A test that starts several sessions' load() concurrently and asserts that every one is served would catch it.

    After one of these aborts, Anthropic children hit a second, permanent refusal with a different fence mismatch. I filed it separately: #794.

  3. magic-alfonso commented on Oct 26, 2026

    @magic-alfonso

    Thanks, the parallel-subagent case is a good addition. The fix covers it: one session's hydration no longer cancels another session's, so first turns that arrive together queue and are each served. We added the test you suggested, which starts five sessions' first turns at once and checks that every one is served with no cancellation. It fails on the 0.49.22 code and passes with the fix. The aborted turns this used to leave behind are what triggered #794, which is tracked separately.

  4. magic-alfonso commented on Oct 27, 2026

    @magic-alfonso

    Fixed in 0.49.24. Thanks @Zireael for the diagnosis, and for the second trigger with several subagents starting at once.

    A session's tag hydration is no longer treated as failed when another session's background preparation is cancelled. A turn may now cancel only a different session's background prewarm that no turn is waiting on, never another turn, and a cancelled prewarm records no failure. The first turn after a restart, and several subagents starting together, are now served. A test starts five sessions' first turns at once and requires all five to be served.

    Update the Pi package and restart Oh My Pi to pick it up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions