Repository navigation
External-system observability: unified PM + queue + quota attribution (webhooks, hot list, session IDs, projections) #2072
Description
Problem (owner direction, 2026-10-28)
Work is steadily moving off the main agent onto other systems that spend their own tokens under direction — the OpenCode fleet, OpenClaw-originated sessions, Codex/ChatGPT, Jules and other external agents. Direction stays centralized (adjust cadences, add/remove scheduled tasks, tell a persistent agent what it is doing right or wrong and let it fix itself), which saves main-agent tokens. The cost: the main agent no longer directly knows what those systems did. Today the record is scattered across GitHub, the queue DB, session stores and quota files, with no single place to answer "what happened, who did it, what did it cost, and was it any good."
This issue is the design + build tracker for closing that gap.
Requirements
1. The PM stack and the queue must be available TO the external systems
Not just readable by the owner: the delegated systems themselves need to read queue state and report outcomes into the PM stack, so their work lands in the shared record as it happens instead of being reconstructed later.
2. Detect what happened OUTSIDE the system
Anything a delegated system does that bypasses the queue/PM record is currently invisible. Close this with webhooks: for every system/repo that can emit them (GitHub first), deliver events into an ingestion point that maintains a hot list of everything created or changed since the last check — issues, PRs, comments, edits, commits. The checking/triage systems then sweep the hot list and flag anything that was not expected (not traceable to a queue item, a schedule, or a directed task).
3. PM stack + queue: one integrated view
Merge or closely integrate the two so the owner can see, in one place:
- what is queued, and in what order;
- what has been picked up, and by which system;
- stats, metrics, graphs and trends over token usage, model selection, plan quotas, and all scheduled jobs.
4. Per-job / per-model outcome attribution
For each job (and model) the system should be able to say: "this job with this model made 2 PRs, closed 1 issue, created 3 issues, commented on these issues" — counting at least: issues/PRs created, commented on, descriptions edited, commits to branches — with every aggregate linking back to the exact issues, PRs and comments behind it.
5. Universal session identity
Every unit of delegated work carries a traceable ID:
- OpenCode already has session + parent-session IDs — use them as-is (session uploads/digests already preserve child-session lists and model switches).
- OpenClaw origination of an OpenCode session needs a recorded mapping (which OpenClaw context started which session).
- Systems whose internals we do not control (ChatGPT/Codex and similar): pull actual IDs from a weekly ChatGPT export; where no native ID exists, workers get an assigned ID when they pick up a queue job and reuse it across subsequent items.
- The same applies to the main agent itself: when it takes an item from the system it is told its session ID (or presents one it already holds), so its work is attributed the same way.
Sessions tie to: exact model(s), the system of origin, token usage where measurable (exact where the platform records it, absent where it cannot), and the quota series (item 7) — so a change to a job can be followed to its effects on quotas and outcomes.
6. Quality linkage
- PRs can be marked reverted or had to be redone, with links to the redo.
- A bug can be tied to the multiple PRs that contributed to it existing.
Together with item 4, this is how we find out what is actually causing bugs — which jobs/models/systems produce work that sticks vs work that gets reverted — and the UI should make that analysis easy.
7. Quotas in the same picture, with projections
- The assistant's own subscription quota joins the PM pool list as a first-class entry: current percentage, total tokens per week, position in the window. (Data source exists: a 5-minute recorder landed 2026-10-28 —
~/tools/quota-recorder.py+ the runtime token DB's per-message records; the laptop quota-gatherer already records the other pools every 15 minutes.) - For every quota: a few days of percentage history against its reset, and a projection — either "projected to end the window at X%" or "projected to hit quota on at ".
- Projections are stored, and later compared with what actually happened, so modifiers/rules can be learned over time to make the predictions better.
Persistence principle
Not everything has to land in the database immediately — but everything must be written down somewhere persistent, with a system that gets it into a central location where it can be queried regularly.
Pieces that already exist (build on these)
- PM aggregator + projector (laptop) with per-source health, comments/actions stores, quota-current and the monthly usage-snapshot JSONL.
- The queue system (Supabase
agent.tasks: packets, dedupe keys, lane claims, completions — completions currently record queue-level worker identity, not per-lane; see the 2026-10-28 token check-in findings). - The runtime token DB (
agent.agent_message_token_usage+ spawn metadata) — exact per-message tokens with cron attribution for everything on this platform. - OpenCode session digests (full session/child-session/model-switch records).
- The 5-minute Muse subscription quota recorder + the laptop's 15-minute pool gatherer.
- GitHub webhooks are available on all owned repos; the fleet-feed collector pattern is a working ingestion precedent.
Shape of the work (to be refined in design)
- Design note first: identity scheme (session/job/system), the attribution event model, webhook ingestion + hot-list store, and the integrated PM/queue/quota views. Record the design on this issue before building.
- Then implementation issues, one per piece, linked here: webhook ingestion + hot list; queue↔PM integration views; attribution rollups (per job/model/session); session-ID assignment at queue pickup + OpenClaw/Codex mapping; projection engine + accuracy tracking; Muse pool on the PM list.
Acceptance criteria
- A design note is recorded on this issue covering: the identity scheme (session/job/system), the attribution event model, webhook ingestion + hot-list store, and the integrated PM/queue/quota views.
- Implementation issues are filed, one per piece, and linked here (webhook ingestion + hot list; queue↔PM integration views; attribution rollups; session-ID assignment at queue pickup + OpenClaw/Codex mapping; projection engine + accuracy tracking; Muse pool on the PM list).
- GitHub webhooks (first source) deliver issue/PR/comment/commit events into the ingestion point, and the hot list reflects everything created or changed since the last check.
- The integrated view can answer, for a given window: what happened, which system/agent did it, what it cost, and its outcome — without hand-reconciling GitHub, the queue DB, and quota files.
Activity
developing-today commentedon Oct 29, 2026
CODE-989
dezren39 commentedon Oct 29, 2026
Intake review
Related: owner-direction tracker filed 2026-10-29 by dezren39; the fleet's existing queue signal/PM stack and the repo-queue helper are the integration substrate; the stewardship/orchestrator hot-list pattern named in the body is the ingestion precedent.
Notes: this is a design-first tracker — step 1 in the body (design note recorded here before building) is the gate; the six implementation pieces should be filed as linked issues only after the design lands, per the fleet's nothing-sits-silently rule.
Risks: scope is fleet-wide (multiple external systems); webhook delivery needs a reachable ingestion endpoint, which may gate the first piece.
Suggested approach: design note first (identity + event model), then file the six linked implementation issues in the order listed in the body.
— Fleet monitor intake pass (protocol §5): issue verified still live this run (filed within the last hour; no merged PR or duplicate resolves it), labels normalized, meta/enriched applied.
dezren39 commentedon Oct 29, 2026
DREW DIRECTION (2026-10-29) — the OpenCode usage exporter: PULL from OpenCode's own history database. We have direct query access to that database and it is backed up on a routine basis; the exporter reads sessions/messages (model, tokens, cost, parentage) out of it on a cadence, the same pull pattern as the Muse runtime-db export — do NOT build push instrumentation into OpenCode/OpenChamber for this. One implementation fact from tonight's lane work: the live session data is in OpenChamber's MANAGED OpenCode instance store — the default-path opencode.db is stale (newest activity ~Oct 6) — so the exporter must target the live instance's database (and ideally notice if the instance's store ever moves). Aster's design (2026-10-29) makes this exporter the precondition for any measured OpenCode baseline: mirror_* tables cover Muse's runtime only.
dezren39 commentedon Oct 31, 2026
Status — owner order 2026-10-28, stalled, re-queued 2026-10-31.
This observability direction was given by Drew on 2026-10-28 (issue priority/high), but the issue never got a queue presence — no task row, no queue labels, no execution evidence in 3 days.
Remediation (2026-10-31, Oct-28 left-on-the-ground audit): enqueued into repo-health-code-pilot as task 4722 (triage/design, priority 70, queued, attempts 0/3), verified by direct queue read; queue signal refreshed.
The packet scopes the triage to what this issue needs first: sequence the seven requirements into landable slices, fully specify the first slice, and route it — not an attempt at the whole build in one task.
50 remaining items
developing-today commentedon Nov 3, 2026
agent-note:v1
state: reviewed (coverage loop-back)
Coverage pass 2026-11-03, loop-back review: this issue was re-reviewed (drift / overlap / already-handled / stale / blockers / ordering) so every open issue in the four repos is reviewed and in the system. Verdict: still live; in the system as task 7285 (triage, queued, priority 50) in repo-health-code-pilot. Labels audited and synced. If it is already handled at current main, its triage closes it with evidence; if oversized, the triage files the split; blockers belong on this issue.
Found via: coverage loop-back 2026-11-03
dezren39 commentedon Nov 3, 2026
🤖 Agent note · picked up
dezren39 commentedon Nov 3, 2026
🤖 Agent note · done
Verified current state of issue #2072:
Routing done by this triage:
Issue #2072 stays open as the tracker; it closes when slices 2–7 land or are individually superseded. Task 7285 completes as
triagedwith follow-up task 8906.