Desktop (Windows): corrupt logs_2.sqlite B-tree causes a deterministic, unrecoverable app-server handshake timeout #39015

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   appIssues related to the Codex desktop appIssues related to the Codex desktop app

   windows-osIssues related to Codex on Windows systemsIssues related to Codex on Windows systems

   app-serverIssues involving app server protocol or interfacesIssues involving app server protocol or interfaces

   on Aug 17, 2026
2. MilkyWay008 commented on Aug 17, 2026

   

   MilkyWay008

   on Aug 17, 2026

   More actions

   Saw your quick_check(3) evidence...... the rowid-out-of-order cells are textbook B-tree corruption, and you're right that WAL replay / checkpoint / reinstall can't fix it because the main DB itself is malformed. The 30s handshake kill ~2s short of completion on every launch is exactly the "structurally cannot self-heal" pattern. Cleanest workaround for end-users right now: fully exit Codex (tray quit, not just window-close...... IDE extensions holding their own app-server also need to stop), then move ~/.codex/logs_2.sqlite + -wal + -shm aside together; rebuilds in ~1.7s and nothing user-authored lives in that file. Genuinely useful framing for the maintainer side: rotating the log DB on SQLITE_CORRUPT instead of looping against a damaged structure would have prevented every one of these outages.
3. jdcodes1 commented on Aug 17, 2026

   

   jdcodes1

   on Aug 17, 2026

   More actions

   Excellent forensics — and the code on main @ 1f41cc5 confirms both halves of your conclusion: the non-essential telemetry DB really is load-bearing for startup, and there is a corruption-recovery mechanism that your failure mode routes around.

   1. logs_2.sqlite failing to open is fatal to the entire state runtime. StateRuntime::init opens the logs pool right after the main state DB, and an error is a hard return Err that tears down the runtime:

   codex/codex-rs/state/src/runtime.rs

   Lines 145 to 155 in 1f41cc5

   	let logs_pool = match sqlite
   	.open_logs_db(&logs_migrator, telemetry_override)
   	.await
   	{
   	Ok(db) => Arc::new(db),
   	Err(err) => {
   	warn!("failed to open logs db at {}: {err}", logs_path.display());
   	close_sqlite_pools(&[pool.as_ref()]).await;
   	return Err(err);
   	}
   	};

   Notably, the post-open logs maintenance a few lines later is already treated as non-fatal (run_logs_startup_maintenance failure only warns, #L261-L266) — so the codebase half-acknowledges that logs are expendable; the open/migrate step just doesn't get the same treatment.

   2. Recovery exists but requires a classified error, not a hang. The app-server wraps init in init_sqlite_state_db_with_fresh_start_on_corruption (app-server/src/lib.rs#L1240+), which — when the error chain classifies as SQLite corruption — moves the damaged file into a backup folder and retries. Your signature B ("failed to initialize sqlite state runtime…", 1 of 6 launches) is init producing an actual error; whether the fresh-start retry then fired for logs_2.sqlite should be visible as backup-folder warnings — if it errored without corruption classification (or attribution), that's a recovery gap worth the log excerpt. But your dominant signature A (5 of 6) never gets there at all: the desktop gives the handshake 30s, and if init is wedged on the corrupt file (SQLite grinding through malformed pages / WAL recovery) rather than failing fast, the process is killed with a generic timeout and the recovery machinery never sees an error to classify. That's why the majority of launches gave "no indication a database was involved" — the diagnosis raced a 30-second clock and usually lost.

   Fix outline:

   1. Demote logs to actually-non-essential: on open_logs_db failure (any error, not just classified corruption), back up and recreate logs_2.sqlite inline and continue — the backup helper used by the fresh-start path (backup_runtime_db_for_fresh_start) already does the file surgery. Telemetry loss is strictly better than a bricked app; the same argument applies to the goals DB one block below.
   2. Bound each DB open/migration step with its own deadline (well under the desktop's 30s handshake budget), so a wedged file becomes a classified, attributable error that the existing recovery machinery can act on — converting your 5-of-6 silent timeouts into the 1-of-6 recoverable path, and then into automatic recovery.
   3. Desktop side: when the handshake times out, surface the app-server's last stderr/error in the modal — signature B proves the actionable message exists; the timeout path just discards it.

   Regression test shape: write garbage pages into a valid-header logs_2.sqlite (rowid-order corruption like yours, not just truncation — open succeeds, scans fail), init the runtime, and assert (a) startup completes with a fresh logs DB and a backup notice, (b) state_5.sqlite is untouched.
4. PollyBot13 commented on Sep 6, 2026

   

   PollyBot13

   on Sep 6, 2026

   More actions

   I encountered the same class of structural corruption in logs_2.sqlite on local macOS APFS, with one important variation: the damaged diagnostic database remained present without a clear Codex recovery event and was first detected by an external full SQLite integrity gate.

   Sanitized environment:

   • macOS on Apple Silicon; CODEX_HOME on local APFS, not NFS/network storage.
   • @openai/codex 0.152.1, launched through an embedding with an agent-specific CODEX_HOME.
   • Database size at preservation: 9,446,199,296 bytes; 4096-byte pages; WAL mode; incremental auto-vacuum.

   Evidence:

   • Full PRAGMA integrity_check returned SQLite error 11, including a freelist leaf-count mismatch, an impossible freelist page number, B-tree initialization failures in root tree 5 (logs), and pointer-map mismatches.
   • A table aggregate failed with database disk image is malformed.
   • Retained backup generations spanning 2026-08-29 through 2026-09-05 showed the same structural error classes and page references. This establishes persistence, not the original corruption mechanism.
   • The database contained diagnostic logs, not the session transcript index. No clear automatic rotate/rebuild event was observed before the external integrity check.
   • sqlite3 .recover --ignore-freelist reconstructed 2,198,392 table rows; 44 timestamp-index-referenced IDs were not recoverable from the table or corroborated fragments. Uncorroborated lost_and_found rows were excluded.
   • The rebuilt database passed full integrity checks and Codex 0.152.1 initialized it successfully.

   Expected behavior: detect structural corruption before normal operation depends on damaged pages, preserve the original and sidecars, rotate or rebuild this non-essential diagnostic store, and emit a clear recovery notice.

   This does not establish that Codex created the corruption, and it does not implicate OpenClaw's backup behavior—the backup refusal was correct. It supports the narrower claim that structurally corrupt logs_2.sqlite can persist without an automatic preservation/rotation path until a full integrity check or damaged-page access surfaces it.