Repository navigation
impl(observability): GitHub webhook ingestion + hot list (#2072 slice 2) #5204
Description
Part of #2072 (external-system observability tracker). Slice 2 of the sequencing recorded on #2072 by the design-triage (task 4722, 2026-11-02). Slice 1 (OpenCode usage exporter) landed 2026-11-03 via PR #4680 / task 6610, which unblocks this slice per the design's ordering.
Scope
GitHub webhook ingestion + hot list (requirement 2 of #2072):
- Deliver GitHub events (issues, PRs, comments, edits, commits) from the owned repos into an ingestion point in the PM/queue stack. Reuse the existing GitHub App substrate (
pm-stack/github-app-manifest.json,github_webhook_secretplumbing — today it delivers to Kaneo, not to any queue/PM ingestion point) rather than building a competing receiver; impl(aggregator): source coverage and gap-aware activity ingestion #3321 explicitly names this ingestion as its dependency and will reuse it. - Maintain a hot list: everything created or changed since the last check, queryable by the checking/triage systems, so anything not traceable to a queue item, a schedule, or a directed task can be flagged.
- Follow the working ingestion precedent:
pm-stack/aggregator/scripts/collect-fleet-feed.ts. - Persistence principle from External-system observability: unified PM + queue + quota attribution (webhooks, hot list, session IDs, projections) #2072: everything written down somewhere persistent, with a path into a central, regularly queryable location. Not everything has to land in the database immediately.
Acceptance criteria
- Design/spec for the receiver + hot-list store recorded (event model, dedupe/idempotency on redelivery, checkpointing) — a design-triage pass first, mirroring how slice 1 was specified on External-system observability: unified PM + queue + quota attribution (webhooks, hot list, session IDs, projections) #2072 before it was built.
- Webhook events for at least developing-today/code arrive at the ingestion point and appear on the hot list with source links and stable IDs.
- The hot list can answer "everything created or changed since " and advances its checkpoint only after a successful sweep.
- Duplicate deliveries are idempotent (no double entries).
Out of scope
- Queue↔PM integration views (slice 3), attribution rollups (slice 4), session-ID assignment (slice 5), quality linkage (slice 6), projections (slice 7) — separate slices of External-system observability: unified PM + queue + quota attribution (webhooks, hot list, session IDs, projections) #2072, filed when their turn comes.
- Push instrumentation inside OpenCode/OpenChamber (Drew's 2026-10-29 direction: pull patterns only).
Activity
dezren39 commentedon Nov 4, 2026
🤖 Agent note · picked up
repo-worker-code), OpenChamber scheduled lane02cafe76-eafd-4d0b-aaa6-2b7a737d68dcdezren39 commentedon Nov 4, 2026
Design triage — webhook ingestion + hot list
Current state (verified 2026-11-04)
Issue #5204 remains open with no linked/open PR. Parent #2072 remains open; slice 1 (#4680) is merged. The dependent #3321 implementation PR #5141 is merged, and its done note keeps the GitHub activity slice ordered after #2072/#5204.
This is not a greenfield receiver. On current main
ac149728a,pm-stack/projectoralready verifies GitHub's HMAC signature and delivery GUID, idempotently applies webhook payloads, projects issues/PRs/comments/reviews/pushes, reconciles from GitHub, and exposes item/comment/link reads (README.md:23-34,77-88,158-171;src/server.ts:18-58).pmp_deliveriesis an outcome audit andpmp_push_eventsstores only delivery/ref/head/before/pusher (schema.sql:138-150); mutable item upserts and comment rows are not a complete append-only change feed. There is no hot-events pagination or consumer checkpoint endpoint. The App manifest still sends its single configured webhook URL to Kaneo (pm-stack/github-app-manifest.json:3-5), so replacing that URL with the projector directly would break Kaneo.Recommended design and boundaries
Reuse the existing projector and its namespaced
pmp_libSQL/Turso store; do not add a competing receiver, a second event store in the aggregator, or make the projector a source of truth. Add an append-only normalized activity ledger with a monotonic sequence cursor, unique stable event IDs, delivery GUID, repository, event/action, source ref and URL, source object ID, actor/timestamps where present, and server receipt time. Record supported issue/PR lifecycle and edits, comments/review activity, and individual commit identities from push payloads. Do not retain raw webhook bodies. Same-delivery retries must be idempotent, while later distinct deliveries remain visible even when they update the same item.Expose bounded ascending pages after an exclusive cursor and capture a fixed high-water cursor for each sweep, so arrivals during a sweep are left for the next one. Maintain checkpoints per named consumer; the consumer acknowledges only after it has successfully processed through the high-water cursor. Checkpoint writes are monotonic and reject cursors beyond the server-issued high-water mark. A failed/partial sweep leaves its prior checkpoint unchanged and safely replays events. The feed is an audit/hot-list projection with source links, not an event-sourced replacement for GitHub.
Keep live delivery wiring as a separate rollout slice: the existing App URL currently targets Kaneo and the projector README says Cloudflare Worker packaging is still follow-up. A later fanout gateway must preserve Kaneo delivery, forward the original body/signature and
X-GitHub-Deliveryto the projector, retry partial failures safely, and only then update the App endpoint/subscriptions (including review events if required). Do not change live App settings or secrets in the first code slice. Once the feed is live, #3321 can consume it rather than building another GitHub receiver.First landable piece routed
Queue task 9234 (
small_fix, priority 50) implements the append-only projector ledger, event write path, bounded/high-water pagination, per-consumer checkpoint read/ack, fixtures/tests for delivery replay, multiple commits, paging and failed sweeps, and the projector contract docs. It explicitly excludes live App URL changes, Cloudflare deployment/fanout and secrets; those remain a later deployment slice.Triage verdict: still live; design recorded and first implementation piece enqueued.
9 remaining items
fix(projector): validate checkpoint high-water against server ledger (#…
feat(projector): append-only hot list + consumer checkpoints (#5204) (#…
dezren39 commentedon Nov 4, 2026
🤖 Agent note · done
high_water_past_server) and GET /hot clamps a caller pin to the server's actual high-water — with regression tests at store and server level; the disposition is replied on the Codex thread (discussion_r4464825509) and Codex round 2 at fc2a4b9 completed with no findings.