Skip to content

pm dashboard /health: per-source lag counts (behind-by-N + last sync) instead of binary live/fixture only #1734

Open

Description

@dezren39

What

The PM dashboard /health page shows each facet/source as live vs fixture, plus feed-section ages. Add a sharper per-source freshness signal: how far behind the source is (a count) and when it last synced, alongside the existing flags.

Model: the General Board's machines table (board torn down 2026-10-28 with Drew's approval, after a comparison found this its one idea worth keeping). Each machine row carried last_heartbeat, last_sync, and replica_max_entry_id; the page computed "behind by N entries" against the primary and flagged staleness when a heartbeat was >3 min old. "Behind by 1, heartbeat 40s ago" is a much sharper signal than a binary live/fixture flag.

Where

  • pm-stack/aggregator /health page + its /api/*.json twin (keep the machine-first contract).
  • Candidate sources: the fleet-feed collector sections (age + item counts already exist), the quota-gatherer's heartbeat file, the projector when deployed.
  • For each source where a meaningful "current vs seen" count exists, publish behindBy + lastSyncAt; where no count exists, keep the current age display and say why (per code#1500's rule: a section must say WHY it is empty/unresolved).

Acceptance

  • /health shows per-source lag (count + last-sync time) for at least the fleet-feed sections and the quota gatherer.
  • The /api twin carries the same fields.
  • No regression to the existing live/fixture facet flags or the one-row health strip on /today (code#1501).

Filed from the General Board teardown comparison, 2026-10-28 (Drew approved filing this as part of the teardown).

Acceptance criteria

Activity

  1. added
    enhancementNew feature or request
    nextOrg Project gate: queued next — orchestrator adds to the board
    on Oct 28, 2026
  2. developing-today commented on Oct 28, 2026

    @developing-today
    Contributor
  3. added
    source/humanFiled directly by a human
    priority/normalOrdinary priority (the default)
    effort/sSmall: expected under 2 hours of work
    app/aggregatorPackage/app: aggregator (pm-stack in this repo)
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    on Oct 28, 2026
  4. dezren39 commented on Oct 28, 2026

    @dezren39
    MemberAuthor

    Intake review

    Related: #1500 (a section must say why it is empty/unresolved), #1501 (one-row health strip on /today — no-regression target), #1288 (Grist freshness is one of the lag signals this surfaces), #1744 (shared schema will change where freshness data lives).

    Notes: Verified still live at intake (2026-10-28T17:45Z): /health still shows binary live/fixture per the body; no PR adds behindBy/lastSyncAt fields. Acceptance criteria section appended formalizing the body's ## Acceptance list; original body preserved above it.

    Risks: Not every source has a meaningful "current vs seen" count — forcing one produces a misleading number, which is worse than the binary flag; the body's escape hatch (keep age display + say why) should be used liberally. Small, well-scoped item (effort/s).

    Suggested approach: Start with the fleet-feed sections and quota gatherer (counts/heartbeat already exist), publish behindBy + lastSyncAt in the /api twin first, then render on /health.

    🤖 Fleet monitor intake enrichment · 2026-10-28

  5. dezren39 commented on Oct 31, 2026

    @dezren39
    MemberAuthor

    Status — owner order 2026-10-28, stalled, re-queued 2026-10-31.

    Drew approved this salvage from the General Board teardown on 2026-10-28. A triage task (task 2632) was queued that evening and is still live — but no implementation packet ever followed, so the work never started.

    Remediation (2026-10-31, Oct-28 left-on-the-ground audit): a small_fix packet is now enqueued into repo-health-code-pilot as task 4723 (priority 50, queued, attempts 0/3), verified by direct queue read; queue signal refreshed.

    Pairing rule recorded in the packet: if triage 2632 has produced a verdict by claim time, the fix follows it; if not, this issue's body + acceptance criteria are the spec (they were filed complete, effort/s) and the triage closes out as superseded-by-fix rather than duplicating the work.

  6. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    agent-note:v1 — holder: repo-worker-code (OpenChamber scheduled lane), since 2026-11-01. Picked up queue task 2632 to revalidate #1734 against current main and check the existing follow-up task referenced in the Oct-31 status note before deciding whether this triage is superseded.

  7. 32 remaining items

  8. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

    • Agent: fleet monitor (repo-fleet-monitor)
    • Session / ID: task 4723 · repo-fleet-monitor run 2026-11-03 08:12 CST
    • Found via: monitor reconciliation
    • Task: Claim-evaporation repair. Task 4723 was claimed and this issue labelled queue/claimed + work/assigned, but the holder's run ended without complete/park/park/block; the lease expired and the task returned to queued, unclaimed. Labels are reset to the truth: queue/queued + status/needs-triage. Last checkpoint stage recorded on the task: handed_off — the next claimant resumes from that checkpoint, it does not restart from zero.
    • Expectation: a queue lane re-claims task 4723 and continues from stage handed_off.
  9. added
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 3, 2026
  10. developing-today commented on Nov 3, 2026

    @developing-today
    Contributor

    agent-note:v1
    state: reviewed (coverage loop-back)
    Coverage pass 2026-11-03, loop-back review: this issue was re-reviewed (drift / overlap / already-handled / stale / blockers / ordering) so every open issue in the four repos is reviewed and in the system. Verdict: still live; in the system as task 4723 (small_fix, queued, priority 50) in repo-health-code-pilot. Labels audited and synced. If it is already handled at current main, its triage closes it with evidence; if oversized, the triage files the split; blockers belong on this issue.
    Found via: coverage loop-back 2026-11-03

  11. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 3, 2026
  12. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    state: picked_up
    worker: repo-worker-code
    task_id: 4723
    issue: #1734
    branch: queue/4723-1734-source-lag-health
    summary: Reclaimed the implementation task. Fresh GitHub state: issue is open, no hard-stop labels or matching open implementation PR; sibling triage 2632 is done and marked superseded. Muse/Drew direction in the issue defines behindBy as provider-reported total minus emitted entries, with null when no total is reported.
    next: verify and resume the existing branch/worktree from the handed_off checkpoint; implement and test this defined metric.

  13. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    state: handed_off
    worker: repo-worker-code
    task_id: 4723
    issue: #1734
    branch: queue/4723-1734-source-lag-health
    pr: #5180
    summary: Added reported-total minus emitted-entry lag where comparable, lastSyncAt and explicit unknown reasons for feed sections and quota-gatherer providers. PR is open; local aggregator suite passes.
    verification: bun test — 1434 passed; git diff --check — passed. Typecheck remains blocked by the unchanged src/item-history.ts:57 GhKind error. No PR CI run appeared; both aggregator workflows are main-push-only, and the coverage gap is documented on the PR.
    next: review/merge and deploy PR #5180, then verify live /health and /api/health fields.

  14. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    state: done
    worker: repo-worker-code
    task_id: 4723
    issue: #1734
    branch: queue/4723-1734-source-lag-health
    pr: #5180
    summary: Queue task completed as pr_opened after verifying the implementation and recording the CI coverage gap on the PR. The issue remains open pending PR review, merge, deployment, and live endpoint verification; queue/work/status labels for this task are cleared.
    next: merge and deploy PR #5180, then verify /health and /api/health against the deployed build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/aggregatorPackage/app: aggregator (pm-stack in this repo)app/pm-stackpm-stack (Pad/Grist) in developing-today/codeeffort/sSmall: expected under 2 hours of workenhancementNew feature or requestkind/enhancementmeta/enrichedIntake review done: acceptance criteria, links, and normalized labels addednextOrg Project gate: queued next — orchestrator adds to the boardpriority/normalOrdinary priority (the default)source/humanFiled directly by a humantask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions