Repository navigation
codex-companion background jobs: no turn timeout, no PID-liveness reaping, shell-mangled taskkill (Windows) — jobs stick as 'running' forever #478
Description
Bug report: codex-companion job runner has no turn timeout, no PID-liveness check, and a shell-mangling taskkill call
Component: openai-codex Claude Code plugin, version 1.0.6, scripts/codex-companion.mjs + scripts/lib/{tracked-jobs,state,process}.mjs
Platform: Windows (win32), jobs launched via the plugin's task --background / rescue code path
Severity: High — a single wedged turn permanently occupies a scheduler slot and is invisible to status as "stuck"; only a process-tree kill + manual state.json surgery recovers it.
Bug 1 — no wall-clock timeout on a tracked job's turn
runTrackedJob() (scripts/lib/tracked-jobs.mjs:142) does:
const execution = await runner();with no timer, AbortController, or any other mechanism racing the runner. If the underlying Codex turn wedges — e.g. a shell child command it spawned mid-turn hangs (stdin wait, dead network call, orphaned handle) — the awaited promise never resolves and the job record never leaves status: "running".
Evidence: a real job was observed frozen at phase: "verifying", with its last recorded progress event being a custom_tool_call whose output never arrived (the child that should have produced it never returned). By the time it was noticed, the thread had accumulated ~57M input tokens — a runaway loop re-feeding stale context with no timeout to cut it off.
Bug 2 — no PID-liveness check when jobs are read back from state.json
listJobs() (scripts/lib/state.mjs:149) returns the jobs array from state.json verbatim:
export function listJobs(cwd) {
return loadState(cwd).jobs;
}There is no check anywhere in the read path that a job reported running/queued still has a live process behind its pid. If the worker process dies (terminal closed, Ctrl+C, crash, reboot) without going through the normal completion path in runTrackedJob, its state.json entry is permanently stuck at running. Because state.json is an on-disk registry independent of any live process, every subsequent status/task invocation rehydrates the dead job and the scheduler continues to treat it as occupying a slot — indefinitely, with no self-healing.
Bug 3 — taskkill invoked with a shell layer by default on Windows
terminateProcessTree() (scripts/lib/process.mjs:57-98) builds the kill command as an argv array:
const result = runCommandImpl("taskkill", ["/PID", String(pid), "/T", "/F"], {
cwd: options.cwd,
env: options.env
});but never passes shell: false. runCommand()'s default (scripts/lib/process.mjs:12):
shell: options.shell ?? (process.platform === "win32" ? (process.env.SHELL || true) : false)resolves to shell: true on win32 whenever process.env.SHELL is unset (the common case for a plain Windows shell). That routes the ["/PID", "<pid>", "/T", "/F"] args through a shell instead of passing them directly to CreateProcess, which is a plausible source of arg-mangling — particularly when the process tree's ancestor is an MSYS/Git Bash shell, which is known to rewrite /-prefixed arguments (/PID → path-like rewriting) unless explicitly suppressed.
Requested fix (already speced against this exact code, not yet merged)
runTrackedJob: racerunner()against a timeout (env-overridable, e.g.CODEX_COMPANION_TURN_TIMEOUT_MS, default 45 min); on breach, write the job record asstatus: "failed"(with an explicit timeout error message) before any process exit handling runs, and clear/unref()the timer on the normal-completion path so it doesn't keep the process alive.- Job-read path (
listJobsor its callers): reconcile liveness before returning — for any job withstatusinrunning/queued, a numericpid, andcreatedAt/startedAtolder than ~15s (grace period for just-launched jobs), check the PID is actually alive; if not, rewrite it tostatus: "failed"in both the per-job file and thestate.jsonregistry. terminateProcessTree: passshell: falseexplicitly on thetaskkillcall so the argv array reachesCreateProcessunmodified.
Reproduction notes
- Bug 2 was independently reproduced by hand: append a synthetic
state.jsonjob entry with a dead numericpid,status: "running", and acreatedAtwell over 15s old, then runstatus— with the current (unpatched)1.0.6code, the synthetic entry is echoed back as stillrunningforever; there is no reconcile step to catch it. - Bug 1 and 3 are confirmed by direct code reading of the shipped
1.0.6source (line references above); no timer/abort logic exists anywhere intracked-jobs.mjs, and no call site inprocess.mjsoverrides the win32 shell default fortaskkill. - All three fixes were speced into a single patch (turn timeout + liveness reconcile +
shell:falsetaskkill, touching onlytracked-jobs.mjs,state.mjs,process.mjs) but a Codex-driven apply attempt against the live plugin install self-blocked on a sandbox write restriction unrelated to the bugs themselves (it could not write its diff-output file outside its task workspace root) — so as of this report the plugin is still unpatched and all three bugs are live in1.0.6.
2026-07-11 update: we applied a local patch implementing exactly the three requested fixes to a deployed 1.0.6 install and verified it: node --check clean, synthetic dead-pid job reaped to failed on status, background-task cancel clean with no orphaned workers.
Activity
axisrow commentedon Jul 14, 2026
Confirming Bug 2 (PID-liveness) from an adjacent layer. I hit the same race in the broker lifecycle and the working shape is: never trust a pid until you've confirmed the endpoint it owns is live. My broker shutdown does
waitForBrokerEndpoint()first, and only tree-kills by pid when the probe succeeded — otherwise an OS-reused pid gets you killing an unrelated process (scripts/lib/broker-lifecycle.mjs). The identical principle drops intolistJobs: before returning arunning/queuedjob, probe its recorded pid (process.kill(pid, 0)), and if dead, flip it tofailedwith aworker_diedreason right there in the read path — no scheduler change needed. Bug 1 (no turn-timeout inrunTrackedJob) is the write-side twin of this; fixing both makes a wedged worker self-heal on the nextstatus. This is the root cause behind #486 and #489 too — worth treating as one surface.