Codex desktop writes high-frequency TRACE logs to ~/.codex/logs_2.sqlite despite RUST_LOG=warn #31111

Activity

1. github-actions

   added

   bugSomething isn't workingSomething isn't working

   appIssues related to the Codex desktop appIssues related to the Codex desktop app

   performance

   on Jul 4, 2026
2. github-actions commented on Jul 4, 2026

   

   github-actionsbot

   on Jul 4, 2026 – with GitHub Actions

   Contributor

   More actions

   Potential duplicates detected. Please review them and close your issue if it is a duplicate.

   • Codex Desktop app-server writes high-volume TRACE rows to logs_2.sqlite despite RUST_LOG=warn and analytics disabled #30988
   • High-frequency TRACE rows are still written to logs_2.sqlite with RUST_LOG=warn #30780
   • macOS Codex 26.623.61825 still persists high-frequency TRACE logs to logs_2.sqlite #30719
   • Codex App writes high-volume TRACE logs to logs_2.sqlite despite RUST_LOG=warn #30236
   • Codex 0.142.0 still writes high-volume TRACE logs into ~/.codex/logs_2.sqlite after restart #29814

   Powered by Codex Action
3. ballack311 commented on Jul 4, 2026

   

   ballack311

   on Jul 4, 2026 via email

   Author

   More actions

   您好  邮件已收到  谢谢
4. Sorkai commented on Jul 5, 2026

   

   Sorkai

   on Jul 5, 2026

   More actions

   I can reproduce the same issue on macOS with the current stable desktop build.

   Environment:

   • Codex Desktop: 26.623.101652
   • Bundle build: 4674
   • Bundled CLI: codex-cli 0.142.5
   • macOS: 26.5.2 (25F84), arm64
   • Writer process: /Applications/Codex.app/Contents/Resources/codex app-server --analytics-default-enabled
   • Local DB: ~/.codex/logs_2.sqlite

   Read-only sampling results. I did not create triggers, run checkpoint/VACUUM, delete rows, or modify the DB.

   Earlier 30s sample during normal interactive use:

   new_rows: 1234
   rows_per_sec: 41.1
   estimated_kib: 2984.59
   estimated_kib_per_sec: 99.49
   
   by level:
   TRACE 1201 97.3% 2569.92 KiB
   DEBUG   16  1.3%  202.56 KiB
   INFO    14  1.1%  175.19 KiB
   WARN     3  0.2%   36.92 KiB
   
   top TRACE targets:
   log                                  978
   codex_api::sse::responses            142
   codex_mcp::connection_manager         48
   codex_app_server::outgoing_message    30
   

   Fresh 20s follow-up sample:

   before_id: 88799455
   after_id:  88799966
   new_rows: 511
   rows_per_sec: 25.6
   estimated_kib: 140.02
   estimated_kib_per_sec: 7.0
   
   by level:
   TRACE 497 97.3% 134.70 KiB
   DEBUG  10  2.0%   3.63 KiB
   INFO    4  0.8%   1.69 KiB
   
   top TRACE targets:
   log                                           475
   hyper_util::client::legacy::pool               8
   hyper_util::client::legacy::client             6
   codex_app_server::outgoing_message             5
   hyper_util::client::legacy::connect::http      2
   codex_api::sse::responses                      1
   

   Representative repeated TRACE fragments include low-level websocket/SSE style entries such as tokio-tungstenite, WouldBlock, Opcode: Data(Text), and app-server delta events. I am intentionally not pasting raw feedback_log_body payloads because the SQLite log DB may contain private session text, local paths, or tool output.

   This looks like the same class of issue as described here: even on 26.623.101652 / 0.142.5, the persistent SQLite sink is still receiving high-volume TRACE rows, and the WAL/main DB mtimes continue to advance during normal use.
5. Sorkai commented on Jul 5, 2026

   

   Sorkai

   on Jul 5, 2026

   More actions

   Additional in-app feedback ID for my reproduction above: 019f3132-544b-7a31-b6f6-485d4c9381b2
6. Gloridust commented on Jul 8, 2026

   

   Gloridust

   on Jul 8, 2026

   More actions

   Root cause analysis, reproduced against current main (f1affbac5e), macOS.

   Two callsites feed the write volume:

   1. http-client/src/transport.rs:99,131 — the transport logs the full request body at TRACE on every model call. That body is the entire prompt context, tool outputs and images included.
   2. codex-api/src/sse/responses.rs:530 — the SSE parser logs every streamed event's payload at TRACE, one row per delta while a turn streams.

   Both sit behind enabled!(Level::TRACE), written on the assumption that TRACE is off in production. But the SQLite sink installed by both the TUI and the app-server (log_db::default_filter()) persists TRACE for all targets, so the guard always passes. This is also why RUST_LOG=warn doesn't help (#29463): RUST_LOG only filters the stderr layer; the sink has its own filter.

   Numbers from my ~/.codex: 87% of retained log content (33 of 38 MiB) is the transport target alone (codex_client::transport in the shipped build, codex_http_client::transport since #31323), with single rows up to 4 MiB. One such row eats most of the 10 MiB per-partition retention budget in a single write and evicts everything else — so this also degrades what /feedback can attach, and it means full prompt content sits unencrypted in logs_2.sqlite.

   Separate but related: the logs db is opened with auto_vacuum=INCREMENTAL, but nothing ever runs PRAGMA incremental_vacuum, so pages freed by retention/pruning stay on the freelist and the file never shrinks below its high-water mark. My logs_2.sqlite is 72 MB with 42% of its pages (30 MB) on the freelist.

   Fixes are small and follow the pattern of #29457 / #26396 / #29599:

   • cap codex_http_client::transport and codex_api::sse at DEBUG in log_db::default_filter(). With no sink enabling those targets at TRACE, the callsites are disabled outright, so the multi-MB serialization is skipped too. RUST_LOG=trace on stderr keeps working for local debugging.
   • run a capped PRAGMA incremental_vacuum(2048) from logs startup maintenance, between the retention delete and the passive checkpoint.

   Both implemented and tested (just test -p codex-state, 153 passed):

   • https://github.com/Gloridust/codex/tree/log-db-drop-http-payload-trace
   • https://github.com/Gloridust/codex/tree/log-db-incremental-vacuum

   Happy to open PRs if the team wants them.
7. dreamitclean commented on Jul 17, 2026

   

   dreamitclean

   on Jul 17, 2026 – with ChatGPT Codex Connector

   More actions

   I reproduced this on macOS 15.3.1 with ChatGPT Desktop 26.715.21425 / bundled codex-cli 0.145.0-alpha.18.

   As a temporary local mitigation (not an upstream fix), I tested a SQLite BEFORE INSERT trigger that drops only TRACE rows while retaining DEBUG, INFO, WARN, and ERROR.

   Before applying it:

   sqlite_sequence delta: +119 IDs / 10 s
   last 60 s: 936 TRACE rows, 3.82 MiB estimated
   

   After applying it during normal interactive use:

   sqlite_sequence delta: +13 IDs / 15 s
   new rows: INFO 7, DEBUG 5, WARN 1, TRACE 0
   codex process-reported bytes written: 352,256 / 15 s
   TRACE rows in a subsequent 5-minute check: 0
   

   A previous active-use sample on the same process was about 12.44 MiB / 20 s, so the observed write volume fell by roughly 96%. The app continued to operate normally in this session.

   Workaround

   Back up the live DB first using SQLite's backup API. Fully quitting Codex first is preferable, but the commands below also use a busy timeout:

   sqlite3 ~/.codex/logs_2.sqlite \
     ".timeout 10000" \
     ".backup '/private/tmp/codex-logs-before-trace-filter.sqlite'"

   Install the filter:

   sqlite3 ~/.codex/logs_2.sqlite \
     ".timeout 10000" \
     "BEGIN IMMEDIATE;
      CREATE TRIGGER IF NOT EXISTS codex_local_drop_trace_logs
      BEFORE INSERT ON logs
      WHEN NEW.level = 'TRACE'
      BEGIN
        SELECT RAISE(IGNORE);
      END;
      COMMIT;"

   Verify:

   sqlite3 ~/.codex/logs_2.sqlite \
     "SELECT type, name FROM sqlite_master
      WHERE type = 'trigger'
        AND name = 'codex_local_drop_trace_logs';"

   Rollback:

   sqlite3 ~/.codex/logs_2.sqlite \
     ".timeout 10000" \
     "DROP TRIGGER IF EXISTS codex_local_drop_trace_logs;"

   Caveats

   • This is an unofficial mitigation and may disappear if a migration/update recreates the logs table or database.
   • It intentionally removes TRACE diagnostics, so users should roll it back before collecting a diagnostic reproduction if maintainers request TRACE data.
   • It stops the SQLite row writes, but it does not prevent the application from constructing the TRACE event before the insert attempt. The proper upstream fix is still to filter the high-volume targets before serialization/enqueueing.
   • I did not delete old rows or VACUUM the live DB. Existing file size is separate from ongoing write churn, and VACUUM would cause a large one-time rewrite.
   • Avoid posting raw feedback_log_body contents; transport rows can contain prompt context, local paths, and tool output.

   This may be useful as a stopgap for affected users while the target-level filtering and incremental-vacuum fixes discussed above are reviewed.
8. etraut-openai

   closed this as a duplicateof High-frequency TRACE rows are still written to logs_2.sqlite with RUST_LOG=warn #30780 on Aug 6, 2026