Repository navigation
[Pi] PiStorageBusyError on SQLite lock contention triggers fail-closed ctx.abort(), blocking interactive turns #601
Copy link
Copy link
Closed
Description
What happened?
In @cortexkit/pi-magic-context (v0.44.4) on Windows 11, when concurrent background maintenance (such as session_project_backfill, dreamer smart-note sweeps, or LKG pruning) holds a write transaction on a large context.db (~2.6GB), any interactive turn that encounters SQLITE_BUSY (database is locked) fails closed instead of failing open:
[magic-context][pi] turn refused Magic Context storage is busy; send your message again
PiStorageBusyError: Magic Context storage is busy; send your message again
at assertPiRawFallbackFits (D:/PiData/agent/npm/node_modules/@cortexkit/pi-magic-context/dist/index.js:28967:11)
...
The user is shown:
Magic Context could not safely prepare this turn; send your message again.
and ctx.abort() is called, immediately cancelling the user's interactive turn.
Reproduction & Log Evidence
- Run Pi with
@cortexkit/pi-magic-contexton Windows 11 with an accumulatedcontext.dbover 1GB. - Background operations start holding transactions:
[magic-context] slow write transaction: site=session_project_backfill held=3208.9ms [magic-context] slow write transaction: site=embedding_stale_gc held=1064.3ms [dreamer] compiled smart-note sweep failed: database is locked - A user message arrives during this lock window. In
assertPiRawFallbackFits:if (!contextLimit || !Number.isFinite(contextLimit) || !observed || ...) { log("raw_fallback_refused completeness=partial"); throw new PiStorageBusyError({ cause }); }
- The error propagates to
registerPiGuardedContext:pi.on("context", async (event, ctx) => { try { return await handler(event, ctx); } catch (error) { log("[magic-context][pi] turn refused", error); if (typeof ctx.abort !== "function") throw error; try { pi.appendEntry(ENTRY_TYPE, { message: RETRY_MESSAGE }); } finally { ctx.abort(); } return { messages: event.messages }; } });
ctx.abort()aborts the active Pi turn.
Expected Behavior
Unlike OpenCode where storage contention fails open (the pass continues or operations defer to the next natural cache bust), in Pi transient PiStorageBusyError should NOT abort the user's turn. It should either:
- Degrade gracefully / fail open to passing raw messages through unmodified (
{ messages: event.messages }) without callingctx.abort(). - Or retry with a brief exponential backoff before refusing.
Technical Analysis & Suggested Fix
- In
packages/pi-plugin/src/pi-context-refusal.ts:
Check the error type inregisterPiGuardedContext. Iferror instanceof PiStorageBusyError(orisTransientPiStorageError(error)), do not callctx.abort(). Instead, log a warning and return{ messages: event.messages }so the user is never blocked from interacting with the agent. - In
packages/plugin/src/features/magic-context/storage-db.ts:
Ensure background maintenance transactions (such assession_project_backfill) are batched into small slices (<100ms) rather than holding continuous multi-second locks on Windows. - Database Maintenance:
Provide a command or automated retention prune to preventcontext.dbfrom growing into multi-gigabyte files that amplify lock contention on Windows.
Environment
- OS: Windows 11 (NTFS)
- Package:
@cortexkit/pi-magic-context@0.44.4 - Harness: Pi 0.99.x
- Client: Pi
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
magic-alfonso commentedon Oct 2, 2026
Thanks for the detailed report and the logs.
The refusal itself is deliberate, and we'll keep it. When storage is busy, Magic Context first tries to resend the last request it built for the session, then retries for the lock, and only then refuses the turn. Passing the unreduced conversation through instead would send everything Magic Context had removed, which can push a long session past the model's context window, and it costs a full prompt-cache rebuild on that turn and again on the next. A refusal costs one resend.
The real problem in your log is that background maintenance holds the database for seconds at a time:
session_project_backfill held=3208.9msandembedding_stale_gc held=1064.3ms. A turn that arrives during one of those holds runs out of retries. Background work shouldn't hold the lock that long, so we're changing these jobs to commit in small batches. We're also checking why your turn reached the refusal instead of resending the last request. The fix is planned for the next release.Two things would help:
session_project_backfillis a one-time job after an upgrade, so if the refusals stop once it finishes, that narrows it down.slow write transactionlines from around those times would tell us which job is responsible.On the database size: the next release also prunes saved requests for sessions idle for more than a day. On our own store that removed about 700 MB.
magic-alfonso commentedon Oct 4, 2026
Fixed in 0.45.0, now on npm. Background maintenance now holds the database for short stretches only: registering embeddings went from about 210 ms to under 2 ms on a 7 GB store, old saved requests are deleted in small pieces, and reading notes no longer takes the write lock. A Pi turn waits at most 250 ms for the database, then sends its last good request again or refuses the turn. As discussed, it never sends the unreduced history.
To update, quit every OpenCode and Pi process first: the database migrates on first start, which can take up to a minute on a large database. OpenCode 1 and Pi pick up the new version on restart. On OpenCode 2 (2.0.22 or newer is now required), open
/plugins, select Magic Context and press ctrl+u, or runopencode plugin update. If you still see this on 0.45.0, please reopen with a log.