Skip to content

Feature: Claude Code-style monitor tool with first-class remote-host support #44855

Open

Description

@abcjhby2871

Summary

Please add a first-class monitor capability to Codex, similar to Claude Code's Monitor tool, with remote-host execution as a core requirement.

There is already a strong proposal for an agent-callable monitor in #29922. The missing piece I want to emphasize is remote development: when Codex is operating against a remote Linux machine / remote workspace, the watcher should run on that remote host, not on the local Mac/desktop process.

This matters a lot for ML / robotics / systems workflows where the expensive work actually runs on remote GPU servers.

Desired behavior

A Codex session should be able to do something like:

monitor(action="start", command="tail -F train.log | grep --line-buffered -E 'OOM|NaN|Traceback|eval finished'", description="training status")

and then:

  1. start the watcher in the currently selected remote execution environment;
  2. let the Codex session become idle without polling or burning turns/tokens;
  3. keep the watcher alive while the remote job continues;
  4. wake the same Codex session when the watcher emits an event;
  5. allow the agent to inspect the new state and continue debugging automatically;
  6. expose list / stop for active monitors;
  7. clearly show whether each monitor is running local or on a particular remote host/session.

Remote-host requirements

For remote workflows, I would like the monitor lifecycle to be tied to Codex's remote execution abstraction rather than to a local shell process.

Concretely:

  • monitor commands should execute on the same remote host/container/workspace where Codex commands are being executed.
  • File watchers should observe the remote filesystem (inotifywait, tail -F, etc.).
  • Process/job watchers should observe remote processes, training jobs, servers, Slurm jobs, Docker containers, etc.
  • A local UI becoming idle should not terminate the remote watcher.
  • Ideally, temporary client disconnects / SSH drops should not silently destroy the watcher; the session should be able to reconnect and recover monitor state when practical.
  • Events should wake the visible Codex session, rather than only a hidden/headless thread. This overlaps with the live-session wake problem discussed in Feature request: deliver external/async events as turns into a live (visible) session — today only headless remote-control threads are reachable #33556.

Example use cases

Remote training

Start the training job on gpu-server-03 and monitor the log.
If OOM / NaN / traceback appears, wake up, diagnose it, patch the config/code, and continue.

Long evaluation

Run the benchmark remotely. Monitor until evaluation finishes, then summarize the metrics and compare them with the previous run.

Remote service debugging

Start the server in Docker on the remote machine and monitor its logs for errors while I work on something else.

Slurm / cluster jobs

Submit the job and monitor `squeue` / job logs. Wake when the state changes to FAILED / COMPLETED or when a target metric appears.

Why polling is not enough

Today an agent can approximate this by repeatedly checking logs/process state, but that is inefficient and fragile for jobs that may take minutes or hours. The desired primitive is event-driven:

remote process / log / file event
        ↓
remote monitor
        ↓
Codex session wakes
        ↓
agent continues reasoning / debugging

This is especially useful for Codex because remote compute is common in software engineering, ML training, robotics simulation, and scientific workloads.

Relationship to existing issues

If maintainers prefer, this can be treated as an extension / acceptance criterion for #29922 rather than a separate implementation track.

Expected UX

The ideal user experience is essentially Claude Code Monitor parity, but with Codex's remote environments integrated cleanly:

Codex: training is running remotely; I am monitoring train.log.

...session is idle...

[monitor: gpu-server-03 / training status]
RuntimeError: CUDA out of memory

Codex automatically wakes and starts diagnosing the failure.

That would make Codex substantially better for long-running remote development and research workflows.

Activity

  1. ysyysy115 commented on Sep 17, 2026

    @ysyysy115

    Very important and useful feature!

  2. abcjhby2871 commented on Sep 24, 2026

    @abcjhby2871
    Author

    Update (2026-09-24): this is becoming a core primitive for long-horizon / research-agent workflows

    I want to sharpen the motivation for this request.

    This is no longer just about “Claude Code Monitor parity”. A first-class remote event-driven wait/wake primitive is increasingly a prerequisite for long-horizon autonomous engineering, ML research, and RSI-style experiment loops.

    The desired execution model is:

    model reasons
        ↓
    launch deterministic / expensive work
        ↓
    model goes fully dormant
        ↓
    remote job / watcher owns the wait
        ↓
    meaningful event occurs
        ↓
    same visible Codex thread wakes
        ↓
    model reasons again
    

    rather than:

    model → poll → model → poll → model → poll → ...
    

    Evidence from Codex itself

    Several recent issues now appear to converge on the same missing runtime substrate:

    That last case is important because it shows this is not only a UX improvement. It is also an inference-efficiency / cost architecture issue: machine progress should not require model inference.

    Why remote execution should be first-class

    For ML, robotics, simulation, scientific computing, and increasingly AI-research-agent workloads, the expensive work is normally not running on the laptop:

    Codex Desktop
          │
          └── remote GPU / Docker / Slurm
                    │
                    ├── train
                    ├── checkpoint
                    ├── eval
                    └── logs / metrics
    

    These jobs can run for hours or days. A useful agent runtime therefore needs to let the watcher live with the remote execution context, survive an idle UI, and resume reasoning only at decision points such as:

    • process exit / failure;
    • OOM / NaN / traceback;
    • checkpoint ready;
    • evaluation complete;
    • metric regression / plateau;
    • Slurm state transition;
    • a user-defined log/file/event condition.

    A local-only watcher tied to the Desktop process would solve only a small subset of this problem.

    The competitive bar is also moving

    Meta's Muse Code now documents several runtime concepts that make this direction especially relevant:

    • persistent asynchronous background agents;
    • goals / long-running loops;
    • background tasks with status/control;
    • workflows that continue in the background and deliver completion back into the conversation without user polling;
    • session messaging and a durable event log.

    References:

    I do not think Codex needs to copy Muse's architecture. But it does mean that “the model can reason for a long time” is no longer sufficient; the runtime has to remain alive cheaply and reactively around long-running external work.

    Why this matters for research / RSI-style loops

    Recent research is also making this workflow concrete rather than hypothetical. For example, AIDE² reports an autonomous 8-day recursive self-improvement run in which an AI research agent repeatedly modifies itself, benchmarks the modified agent, and keeps successful changes:

    https://arxiv.org/abs/2609.26457

    Regardless of whether we call a particular workflow “RSI”, the systems pattern is the same:

    hypothesis / code change
            ↓
    long training or benchmark
            ↓
    evaluation artifact
            ↓
    decision
            ↓
    next experiment
    

    The model should be active at the decision boundaries, not continuously burning inference while the GPU/cluster is doing deterministic work.

    Suggested acceptance criteria

    I would treat #29922 (monitor), #47193 (live-session wake), and this issue (remote semantics) as pieces of one substrate.

    A strong implementation would ideally provide:

    1. Remote binding — a monitor launched from a remote Codex workspace runs on that same host/container/workspace.
    2. True dormancy — while the watcher is quiet, there are no model calls / reasoning turns.
    3. Same-thread wake — an event resumes the same visible Desktop/CLI thread and, if applicable, its active Goal.
    4. Safe delivery — if the model is already in a turn, events are queued/coalesced rather than interrupting or spawning duplicate reasoning.
    5. Reconnect semantics — temporary Desktop disconnects / SSH drops should not silently lose watcher state; persistence would be ideal, but at minimum loss should be explicit and recoverable.
    6. Structured event metadata — monitor id, host/workspace, timestamp, source, exit status / matched condition, and bounded output.
    7. Control surface — list / pause / resume / stop monitors, with an obvious local-vs-remote indicator.
    8. Common sources — process completion, stdout/stderr/log matching, file changes, and arbitrary shell-based watchers so Slurm/Docker/training systems do not need bespoke integrations.
    9. Autonomy guardrails — retry/action budgets and bounded self-repair loops, so event-driven automation cannot silently turn into an unbounded repair cycle.

    An optional higher-level layer could support semantic events such as:

    CHECKPOINT_READY
    EVAL_COMPLETE
    METRIC_PLATEAU
    OOM
    NAN
    JOB_FAILED
    TRAIN_COMPLETE
    

    but the core shell/process monitor + durable wake primitive would already make those easy to build.

    Example research loop

    Codex Goal: improve validation success rate
            ↓
    edit code/config
            ↓
    submit training on remote GPU
            ↓
    monitor goes quiet; Codex sleeps
            ↓
    EVAL_COMPLETE {score: 0.78}
            ↓
    Codex wakes, inspects artifacts, decides next experiment
            ↓
    submit next run
            ↓
    sleep again
    

    To me, this is the important distinction:

    Long-horizon autonomy should mean a persistent runtime with sparse, event-triggered reasoning — not continuous model polling.

    With Remote SSH now a major Codex workflow, making this primitive remote-aware from the beginning would be disproportionately useful for research and engineering users.

  3. abcjhby2871 commented on Sep 24, 2026

    @abcjhby2871
    Author

    Additional UX / lifecycle requirement: a unified Monitor Manager

    One more product-level requirement seems important if monitors become persistent and remote-aware: there should be a single place to inspect and manage every active monitor.

    Without this, persistent monitors can easily become invisible background state. That creates two problems:

    1. Resource leaks / forgotten watchers — a remote watcher may keep running after the user has mentally moved on from the task.
    2. Unexpected future wakeups / usage — an old monitor may fire hours or days later and trigger model work the user no longer wants.

    A useful Monitor Manager could show, at minimum:

    • monitor name / description;
    • status: active / paused / fired / stopped / disconnected;
    • source: process / log / file / shell watcher / scheduler;
    • execution target: local / remote host / container / workspace;
    • owning thread / Goal;
    • created time and last event time;
    • trigger condition;
    • whether the watcher itself is consuming compute/resources;
    • wake policy;
    • expiration / TTL, if configured.

    And provide bulk controls such as:

    • pause / resume;
    • stop;
    • stop all monitors for a thread / workspace / host;
    • change expiration / TTL;
    • rebind a monitor to another thread;
    • inspect recent events / wake history.

    A small persistent badge/count in Desktop would also help surface "you still have N active monitors", especially before quitting the app or disconnecting from a remote machine.

    Wake destination should be configurable: original thread vs new thread

    I also think monitor delivery should explicitly support both modes:

    1. Wake the original thread

    Best for a tightly coupled long-running workflow:

    thread A
      ↓
    launch training
      ↓
    monitor
      ↓
    event
      ↓
    resume thread A with full context
    

    This is ideal when the event is clearly part of the same Goal / experiment.

    2. Spawn a new thread from the event

    Best when the original thread should remain clean, is already busy, or when the event should be handled independently:

    thread A launches monitor
              ↓
           event fires
              ↓
       create thread B
              ↓
    B receives:
    - monitor metadata
    - triggering event
    - bounded relevant context
    - link/reference back to thread A
    

    This could be especially useful for:

    • multiple concurrent experiments;
    • long-running training across several remote hosts;
    • CI / regression watchers;
    • independent failures that should not interrupt the main reasoning thread;
    • delegating event handling to a cheaper/faster model or subagent.

    A simple policy might be something like:

    wake_target = original_thread | new_thread
    

    with an optional "ask on fire" / routing rule later.

    For long-horizon agent systems, this makes monitors feel less like hidden shell processes and more like first-class persistent objects with ownership, lifecycle, routing, and visibility.

    I would consider the unified management surface and explicit wake routing part of the acceptance criteria, not just nice-to-have UI polish.

  4. sarinstein-yan commented on Oct 18, 2026

    @sarinstein-yan

    +1, especially for research workflows combining remote HPC jobs and queued quantum-hardware experiments. Deterministic watchers should run near the jobs and wake the original Codex thread only on completion or failure. Remote reconnect handling and zero model calls while nothing changes would make this much more useful than periodic AI checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions