Excessive SQLite WAL writes during streaming due to TRACE logs ignoring RUST_LOG

[piotrkacala]

What version of the IDE extension are you using?

openai.chatgpt-26.406.31014-linux-x64 (VSCodium on Linux Mint)

What subscription do you have?

ChatGPT Plus

Which IDE are you using?

VSCodium (Code - OSS fork)

What platform is your computer?

Linux x64 (Linux Mint)

What issue are you seeing?

During model streaming responses, the Codex app-server binary writes ~5 MiB/s (observed up to ~16 MiB/s in iotop) to ~/.codex/logs_2.sqlite-wal. This results in sustained disk write rates that can significantly impact SSD longevity and system performance.

This issue is independent of the thread-stream-state-changed warning spam and the open-in-targets retry loop, which were investigated separately.

Root cause

The extension correctly sets RUST_LOG=warn when spawning the app-server process — confirmed via /proc/<pid>/environ:

RUST_LOG=warn

However, the SQLite logging sink appears to bypass or not respect the RUST_LOG filter, since TRACE-level entries are still persisted to logs_2.sqlite despite RUST_LOG=warn. The SQLite logging path appears to operate outside of the standard RUST_LOG filtering mechanism, or uses a separate filter configuration.

This was confirmed via strace during streaming:

pwrite64(20</home/k/.codex/logs_2.sqlite-wal>, ..., 4096, ...) = 4096
pwrite64(20</home/k/.codex/logs_2.sqlite-wal>, ..., 4096, ...) = 4096
...

And via SQLite query — MAX(id) jumped by ~5000 rows for a single short (~50 token) response, while total row count remained stable, indicating that rows are being rapidly inserted and pruned/rotated during streaming:

-- Before response:
SELECT MAX(id), COUNT(*) FROM logs;  -->  3088667 | 30141

-- After response:
SELECT MAX(id), COUNT(*) FROM logs;  -->  3093673 | 30141

Log level breakdown in logs_2.sqlite:

TRACE  20485  (68%)
INFO    8286
DEBUG   1326
WARN      44

Sample TRACE entries written during streaming:

target=hyper_util::client::legacy::pool   feedback_log_body="idle interval checking for expired"
target=log                                feedback_log_body="WouldBlock"
target=log                                feedback_log_body="/home/runner/work/codex/codex/.cargo-home/git/checkouts/tokio-tungstenite-..."

These are low-level internal network events from tokio-tungstenite and hyper_util with no diagnostic value for end users. They should not be persisted to disk in a production binary — and critically, they are not suppressed by RUST_LOG=warn as one would expect.

What steps can reproduce the bug?

1. Open a workspace in VSCodium/VS Code with the Codex extension.
2. Confirm RUST_LOG=warn is set:

cat /proc/$(pgrep -f 'codex.*app-server')/environ | tr '\0' '\n' | grep RUST_LOG

3. Run strace -f -e trace=pwrite64 -p $(pgrep -f 'codex.*app-server') in a separate terminal.
4. Ask any question in the Codex chat panel and observe streaming.
5. Observe continuous pwrite64 calls to logs_2.sqlite-wal at ~5 MiB/s despite RUST_LOG=warn.

Alternatively, check log level distribution after a session:

sqlite3 ~/.codex/logs_2.sqlite "SELECT level, COUNT(*) FROM logs GROUP BY level ORDER BY COUNT(*) DESC"

What is the expected behavior?

RUST_LOG=warn should suppress TRACE and DEBUG output everywhere, including the SQLite logging sink. The current behavior — where RUST_LOG=warn suppresses stderr output but does not affect SQLite persistence — is inconsistent and undocumented.

TRACE-level events from internal networking libraries (tokio-tungstenite, hyper_util) should never be written to the persistent SQLite log database during normal operation.

Additional information

Suggested fix

The SQLite logging sink in app-server should apply the same log-level filter as RUST_LOG (or the internal tracing subscriber filter), rather than maintaining a separate or hardcoded verbosity level. The fix should ensure that setting RUST_LOG=warn — as the extension already does — is sufficient to suppress TRACE/DEBUG writes to logs_2.sqlite.

Workaround

Users can redirect logs_2.sqlite to tmpfs to protect their SSD while waiting for a fix:

# stop VSCodium first
mv ~/.codex/logs_2.sqlite ~/.codex/logs_2.sqlite.bak
ln -s /tmp/logs_2.sqlite ~/.codex/logs_2.sqlite

Note: logs_2.sqlite contains only internal diagnostic logs — no conversation history — so data loss on reboot is acceptable.

After symlinking logs_2.sqlite to tmpfs, disk I/O from the Codex extension dropped to zero during streaming, confirming that the SQLite WAL is the sole source of the observed write load.

Workaround 2: block inserts into the SQLite log table

Another workaround was suggested by @synap5e in this comment:
#17320 (comment)

Instead of redirecting logs_2.sqlite to tmpfs, users can prevent new rows from being inserted into the logs table by adding a SQLite trigger:

sqlite3 ~/.codex/logs_2.sqlite "CREATE TRIGGER IF NOT EXISTS block_log_inserts BEFORE INSERT ON logs BEGIN SELECT RAISE(IGNORE); END;"

This causes attempted inserts into the logs table to be ignored without returning an error to the application, which should prevent logs_2.sqlite-wal from growing due to persistent TRACE/DEBUG log writes.

To remove the workaround:

sqlite3 ~/.codex/logs_2.sqlite "DROP TRIGGER IF EXISTS block_log_inserts;"

As with the tmpfs workaround, this is not a proper fix. It disables local persistent logging rather than fixing the underlying issue: the SQLite logging sink appears to ignore RUST_LOG=warn and continues persisting TRACE/DEBUG entries during normal streaming.

[christopherball]

I haven't dug into this issue with the same depth as you have, but I started noticing HUGE swings in my disk use over the past month (like 60GB less by end of a single day just doing light-weight work with codex). I eventually realized the "Code Helper" under "disk" within the Activity Monitor on MacOS was showing insane amounts of data being written to disk. This was happening even when I wasn't actively using VSCode (just remained open from past completed Codex work). In my particular case, even after closing VSCode and restarting, the disk space wasn't reclaimed.

It took some extra time to realize this bug was perfectly paired with the default setting of Time Machine holding onto local hourly snapshots on disk so I went ahead and did a backup, manually deleted all local snapshots, and set Time Machine to manual so as to avoid compounding this particular Codex bug.

Given its current state, I am literally going to stop using Codex for now, as it aggressively writes to disk all the time and there's nothing I can do about it. Apologies if what I've described isn't an exact match, but given the overlap of the extension writing to disk at aggressive rates, it felt related.

[clemisch]

I also see this behavior. Do you still have this problem? Could you fix it by symlinking to a tmpfs? Just linking logs_2.sqlite to tmpfs didn't solve the apparent disk writes for me (unless Linux reports writes to tmpfs equally??)

I searched for similar issues and there are mutliple, but no reaction at all from the devs yet. These issues are a month old already 😬

[piotrkacala]

Could you fix it by symlinking to a tmpfs?

It's a workaround but yes, it works. Just make sure that your /tmp is using RAM. Some distros have it set to HDD/SDD.

[clemisch]

Thanks. I ended up just symlinking the whole ~/.codex folder (and copying it back before shutdown, hopefully...). /tmp is indeed a tmpfs for me.

[niklassheth]

I also found this while investigating Codex's CPU usage, writing these trace level logs burns a lot of CPU. A way to reduce the logging level should be added to speed up Codex on older devices.

[synap5e]

sqlite3 ~/.codex/logs_2.sqlite "CREATE TRIGGER IF NOT EXISTS block_log_inserts BEFORE INSERT ON logs BEGIN SELECT RAISE(IGNORE); END;"

[niklassheth]

sqlite3 ~/.codex/logs_2.sqlite "CREATE TRIGGER IF NOT EXISTS block_log_inserts BEFORE INSERT ON logs BEGIN SELECT RAISE(IGNORE); END;"

I tested this and it helps a lot, but there's some more CPU savings from disabling the logging fully.

Each Codex instance was prompted to edit a json file.

Mode	Log rows	Log DB size	time CPU	perf user task-clock	SQLite worker
Full inserts	1888	~3.8 MB	3.64s	2.394s	0.657s
Insert ignored	0	~40 KB	2.42s	1.576s	0.061s
Fully disabled	0	~40 KB	2.01s	1.061s	0 samples

[clemisch]

but there's some more CPU savings from disabling the logging fully.

how did you do that?

[niklassheth]

I patched the codex source to remove the log db's initialization.

Patch: disable log DB initialization

diff --git a/codex-rs/app-server/src/lib.rs b/codex-rs/app-server/src/lib.rs
--- a/codex-rs/app-server/src/lib.rs
+++ b/codex-rs/app-server/src/lib.rs
@@ -600,7 +600,7 @@ pub async fn run_main_with_transport_options(
 
     let feedback_layer = feedback.logger_layer();
     let feedback_metadata_layer = feedback.metadata_layer();
-    let log_db = state_db.clone().map(log_db::start);
+    let log_db = None;
     let log_db_layer = log_db
         .clone()
         .map(|layer| layer.with_filter(Targets::new().with_default(Level::TRACE)));
diff --git a/codex-rs/tui/src/lib.rs b/codex-rs/tui/src/lib.rs
--- a/codex-rs/tui/src/lib.rs
+++ b/codex-rs/tui/src/lib.rs
@@ -1047,7 +1047,7 @@ pub async fn run_main(
 
     let otel_tracing_layer = otel.as_ref().and_then(|o| o.tracing_layer());
 
-    let log_db = state_db.clone().map(log_db::start);
+    let log_db = None;
     let log_db_layer = log_db
         .clone()
         .map(|layer| layer.with_filter(Targets::new().with_default(Level::TRACE)));