Repository navigation
Closed
Labels
app/codex-proxybin/codex-proxy.mjs fleet proxybin/codex-proxy.mjs fleet proxyeffort/sSmall: expected under 2 hours of workSmall: expected under 2 hours of workkind/bugSomething isn't working (prefixed form of the classic bug label)Something isn't working (prefixed form of the classic bug label)priority/highImportant and time-sensitiveImportant and time-sensitivesource/sweepFiled or worked by an issue-sweep jobFiled or worked by an issue-sweep jobstatus/in-progressQueue worker lifecycle stateQueue worker lifecycle state
Description
Observed 2026-10-29 on amd (laptop) by Muse, while verifying the Codex pool after a fresh codex login.
Symptom
After codex login refreshed ~/.codex/auth.json (2026-10-30T02:51:23Z), the laptop codex-proxy's built-in 5-minute syncCodexTokensToOpenCode did NOT update the OpenCode credential store. The active openai credential row still held a stale token (matching neither the old nor the fresh Codex token) until it was synced by hand. The proxy process runs the current script (since Oct 28), syncs at startup and every 5 minutes, and showed no error anywhere — the failure is invisible by design.
Root cause (diagnosed + reproduced 2026-10-29)
bin/codex-proxy.mjs, syncCodexTokensToOpenCode:
execSync(`python3 -c ${JSON.stringify(script)}`, { stdio: 'ignore' });- The sync can never run.
JSON.stringify(script)encodes the Python script's newlines as literal two-character\nsequences. The shell, inside double quotes, does not turn\ninto a newline, sopython3 -creceives the whole script as one physical line full of literal\n. Python dies instantly withSyntaxError: unexpected character after line continuation character. Reproduced in isolation against a scratch sqlite DB with the exact construction from the proxy: same SyntaxError, DB unchanged. - The failure is swallowed twice.
stdio: 'ignore'discards the child's stderr, and the surroundingcatch (err) { // Non-fatal background sync }discards the throw. A failed sync is indistinguishable from a successful one — no log line either way. - Latent overreach (would bite once the sync works): the embedded SQL selects
WHERE integration_id="openai"with noactivefilter and updates everyoauthrow it finds. On the live DB that is TWO rows: the active base row (label='OpenAI',active=1) and an inactive second row (label='OpenAI 2',active=0, untouched since Oct 5). The sync must only ever touch the active baseopenairow — never the inactive row, neveropenai.alternate.
Fix
- Run Python via
execFileSync('python3', ['-c', script])(argv, no shell) so the script arrives verbatim; pass the tokens + DB path via environment variables instead of interpolating them into the Python source. - Restrict the update to
integration_id='openai' AND active=1(plus the existingtype='oauth'check in the row value). - Make outcomes visible: log success (
[codex-proxy] codex-token-sync: updated N credential row(s)) and log failures to stderr with a clearcodex-token-sync FAILED:marker including the child's stderr. Never swallow silently again. - Regression tests in
bin/codex-proxy.test.mjsusing the existing fake-HOME harness: seed~/.codex/auth.json+ a scratchopencode.db, start the proxy, assert only the active baseopenairow was updated (inactive +openai.alternaterows untouched) andtime_updatedadvanced.
Related
- Laptop (amd): proxy systemd units churn with EADDRINUSE while unmanaged copies serve the ports #916 — the serving proxy is the supervisor's transient
fps-codex-proxy-18744unit whilecodex-proxy.servicechurns on EADDRINUSE; deploy/restart for this fix goes through the fps unit / supervisor path, not the churn unit.
Acceptance
- Sync verified live on amd: after a proxy restart, the active
openairow'stime_updatedadvances and its access/refresh hashes match~/.codex/auth.json. -
GET /api/quota/codexstill returnsok:trueafter the roll. - A deliberately broken sync (e.g. unreadable DB) produces a visible
codex-token-sync FAILEDlog line in the proxy journal.
Metadata
Metadata
Assignees
Labels
app/codex-proxybin/codex-proxy.mjs fleet proxybin/codex-proxy.mjs fleet proxyeffort/sSmall: expected under 2 hours of workSmall: expected under 2 hours of workkind/bugSomething isn't working (prefixed form of the classic bug label)Something isn't working (prefixed form of the classic bug label)priority/highImportant and time-sensitiveImportant and time-sensitivesource/sweepFiled or worked by an issue-sweep jobFiled or worked by an issue-sweep jobstatus/in-progressQueue worker lifecycle stateQueue worker lifecycle state
Activity
fix(codex-proxy): repair Codex->OpenCode token sync (#2533)
dezren39 commentedon Oct 30, 2026
Diagnosis update (2026-10-29, fixing agent): a SECOND independent failure layer was found while preparing the deploy, and the fix approach changed accordingly.
Layer 2: there is no
python3on the deployed proxy's PATH at all. The serving process (fps-codex-proxy-18744, PID 3133249) runs withPATH=/run/current-system/sw/bin:/home/user/.local/bin:/usr/bin:/bin. Verified live:python3exists in none of those (/run/current-system/sw/bin/python3,~/.local/bin/python3,/usr/bin/python3,/bin/python3all absent — on NixOS it lives at/etc/profiles/per-user/user/bin/python3, which the service PATH does not include). So even with the quoting bug fixed, the Python child could never have spawned in the deployed environment.Fix as implemented in PR #2538: the sync no longer uses a child process at all — it runs in-process via
node:sqlite(DatabaseSync, lazilycreateRequired so a missing module degrades to a logged failure, never a dead proxy). This removes the quoting class, the PATH/interpreter dependency, and all token interpolation. Scope (active baseopenairow only) and visibility (success line +codex-token-sync FAILED:stderr marker) are as specified in the issue. Both new regression tests pass on the laptop (~0.4s).CI note: repo-wide
nix cirunner stall (all runsqueuedsince ~2026-10-30T03:07Z) — the required check's script was run verbatim against the branch on the laptop and passes; PR is in the merge queue. Live deploy + sync verification results will be noted here when done.dezren39 commentedon Oct 30, 2026
FIX DEPLOYED + VERIFIED LIVE (2026-10-29 ~22:47–22:53 CDT). PR #2538 (head 82d94fa) is in the merge queue; repo-wide GitHub runner stall (every
nix cirunqueuedsince ~03:07Z) is the only thing between the PR and merge — the required check's script was run verbatim against the branch on the laptop and passes. Because the fix-it-now order does not wait on stalled runners, the deploy was landed from the pushed branch content and is hash-verified identical (primarybin/codex-proxy.mjssha256 dad5f798… == branch blob). When the queue merges, the primary checkout becomes a no-op.Deploy actions:
~/git/code:git checkout origin/fix/2533-codex-token-sync -- bin/codex-proxy.mjs bin/codex-proxy.test.mjs(surgical; pre-existing flake.lock/sessions dirt untouched).codex-proxy.service(Laptop (amd): proxy systemd units churn with EADDRINUSE while unmanaged copies serve the ports #916) so it could not steal port 18744 mid-restart, thensystemctl --user restart fps-codex-proxy-18744— the serving transient unit only. OpenChamber was NOT restarted. New MainPID 3968988; proxy-supervisor adopted it (state hash aa3bb7c9…, adopted:true). Note:codex-proxy.servicelater resumed its auto-restart churn on its own (another actor re-started it) — it holds no port and does not affect the fps instance; see Laptop (amd): proxy systemd units churn with EADDRINUSE while unmanaged copies serve the ports #916.Verified sync cycles (the thing that never worked before):
[codex-proxy] codex-token-sync: updated 1 credential row(s).cred_10b5fd438001thsQzMXLQMc9oD: time_updated 1793328924205 (2026-10-30T02:55:24Z, the earlier hand sync) -> 1793332031557 (03:47:11.557Z); access sha256/12 97d43ababf95 and refresh 307b668ff3db — both exactly match~/.codex/auth.json; expires = time_updated + 30d.OpenAI 2(active=0) andopenai.alternaterow: timestamps and hashes byte-identical before/after — untouched, as required.[codex-proxy] codex-token-sync FAILED: file is not a databaseat 22:53:14 CDT (probe unit stopped and removed; live data untouched). The corrupt-DB regression test asserts the same marker.GET /api/quota/codexactive entry ok:true, weekly usedPercent 34, account drewrypope@gmail.com (baseline before restart was ok:true at 33%). Proxy /health ok on 18744.Remaining: merge of PR #2538 when runners recover (auto-merge/queue already armed; it will auto-close this issue via
Fixes #2533).