Skip to content

id: build: grant vesting reconciler and control-plane↔ledger drift alarm #2295

Open

Description

@dezren39

Follow-up from the #2065 design (doc/2026-10-29T12-10-00Z_design_refundable_grants_pending_transfers/, PR #2293), §6.3.

Why

The pending-transfer mechanism adds failure modes the posted design did not have: a service that dies between reserve and vest leaves the user's paid grant to expire on the TigerBeetle timeout, and a crash between writing intent and submitting the transfer leaves a row whose state disagrees with the ledger.

Scope

  • Sweeper: state=pending AND window_ends_at < now → post (retry with backoff), then mark posted; Expired is a reconcilable state, not a loss.
  • Safety net: timeout = window + slack (design recommends +72h) so a dead service cannot hold a grant forever; the sweeper must post before that, and the slack is a last resort.
  • Drift alarm: compare each grant's pending_transfer_id resolution in TigerBeetle against the row state; mismatch is an alert (never a silent fix).
  • Late-refund fallback per §6.2: clawback attempt, on exceeds_credits mark the grant debt and expose it to the gateway's grant/redemption refusal.
  • Metrics: pending count, oldest pending age, expired-then-reconciled count, drift count, debt total.

Depends on

The grant service issue (creates the table and endpoints this reads).

Done when

A test cluster with a killed service between reserve and vest self-heals via the sweeper, a seeded drift row raises an alarm, and an after-vest refund lands in debt rather than throwing.

Acceptance criteria

  • Sweeper posts pending grants past window end (retry with backoff) and marks them posted; Expired is handled as reconcilable, never a silent loss
  • Drift alarm fires on a seeded row whose TigerBeetle pending_transfer_id resolution disagrees with the row state; mismatches alert, never auto-fix silently
  • After-vest refund path lands the grant in debt and exposes it to gateway refusal (no throw)
  • Metrics exist: pending count, oldest pending age, expired-then-reconciled count, drift count, debt total
  • Killed-service test: a service dying between reserve and vest self-heals via the sweeper

Activity

  1. developing-today commented on Oct 29, 2026

    @developing-today
    Contributor
  2. added
    priority/normalOrdinary priority (the default)
    effort/mMedium: expected under 1 day of work
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    on Oct 29, 2026
  3. dezren39 commented on Oct 29, 2026

    @dezren39
    MemberAuthor

    Intake review

    Related: #2065 design (PR #2293, §6.2/§6.3), the grant-service build issue #2294 (creates the table this reads), epic #2056

    Notes: Follow-up from the #2065 design; depends on #2294's grant table/endpoints existing first.

    Risks: The +72h timeout slack is a last resort — a sweeper that routinely relies on it is masking a dead service.

    Suggested approach: Build after #2294; test-first with the killed-service and seeded-drift scenarios from the body's Done-when.

    🤖 Agent note · filed

    • Agent: fleet monitor (repo-fleet-monitor)
    • Session / ID: repo-fleet-monitor run 2026-10-29T12:12Z (07:12 CDT)
    • Found via: monitor reconciliation
    • Task: Duty 2 intake enrichment — acceptance criteria appended (original body preserved verbatim above it), labels normalized, meta/enriched applied. Still-live check this pass: no merged PR or fix commit explicitly resolving this issue was found; where a queue triage task owns verification, it is named in the notes.
    • Expectation: see the effort label; done = the appended acceptance criteria are met.
    • Links: this issue
  4. added
    queue/queuedHas a queued task in the Supabase agent queue
    source/creatorFiled by the new-issue-creator job
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 3, 2026
  5. developing-today commented on Nov 3, 2026

    @developing-today
    Contributor

    agent-note:v1
    state: triaged (coverage pass)
    Coverage pass 2026-11-03: reviewed in the fleet-wide per-issue pass (drift / overlap / already-handled / stale / blockers / ordering). Verdict: still live; queued in repo-health-code-pilot as triage at priority 40 (priority/normal). If this is already handled at current main, the triage closes it with evidence; if it needs splitting, the triage files the split.
    Found via: coverage pass 2026-11-03

  6. 18 remaining items

  7. added
    queue/queuedHas a queued task in the Supabase agent queue
    and removed
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    on Nov 4, 2026
  8. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

    • Agent: fleet monitor (repo-fleet-monitor)
    • Session / ID: task 7866 · repo-fleet-monitor run 2026-11-03 20:12 CST
    • Found via: monitor reconciliation
    • Task: Claim evaporated — task 7866 returned to queued and unclaimed in repo-health-code-pilot after the holding run ended without a terminal verb (checkpoint stage: handed_off). Labels reset to the truth: queue/queued + status/needs-triage. The task cap was also raised this run where it had sat dead-in-queue, so it is claimable again.
    • Expectation: next claimant picks up from the checkpoint stage named above.
  9. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    event: picked_up
    task: 7866 (triage, repo-health-code-pilot; worker repo-worker-code; OpenChamber scheduled lane 7b47536b-c162-444c-be8a-3a9538cc4cb5)
    by: openchamber-scheduled-lane
    at: 2026-11-04T04:16Z

    Claimed task 7866 and resumed its prior handed-off state. I will verify issue #2295, current implementation/PR state, and any active owner before routing or closing the triage.

  10. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    on Nov 4, 2026
  11. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    event: done (triaged; implementation dependency-blocked)
    task: 7866 (triage, repo-health-code-pilot; worker repo-worker-code; OpenChamber scheduled lane 7b47536b-c162-444c-be8a-3a9538cc4cb5)
    by: openchamber-scheduled-lane
    at: 2026-11-04T04:19Z

    Fresh verification: issue #2295 remains OPEN; no matching implementation PR exists. Current origin/main is 12fcad9, and no grant-vesting reconciler or grant-service implementation was found under pkgs/id.

    Dependencies and ordering:

    1. id: build: TigerBeetle cluster provisioning — RTT mesh gate, 6-replica format, dead-replica runbook + rehearsal #2292 remains OPEN and formally blocked/escalated (queue/blocked, status/blocked, user/escalated) because its RTT mesh and 6-replica TigerBeetle rehearsal require Drew-provided cloud accounts and a spend cap. Its recorded escalation names the access and cost decisions required.
    2. id: decide: grant refund window length and whether pending tokens are spendable #2296 remains OPEN in user/review; this is Drew's policy decision on the refund window and whether pending tokens are spendable. Active triage task 7322 owns that decision; do not choose a default here.
    3. id: build: grant service — pending-transfer grants (register/vest/void) with control-plane grant records #2294, the grant service/table/API prerequisite, remains OPEN and unbuilt. Its existing triage records those same prerequisites.
    4. id: build: grant vesting reconciler and control-plane↔ledger drift alarm #2295's reconciler/drift alarm follows id: build: grant service — pending-transfer grants (register/vest/void) with control-plane grant records #2294. Scope remains as written in this issue.

    No duplicate implementation task was enqueued while the upstream service and its blockers remain unresolved. Task 7866 is complete as triaged. Transient lifecycle labels set by this task were removed; issue classification labels remain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/idPackage/app: id (pkgs/ or apps/ in this repo)effort/mMedium: expected under 1 day of workkind/enhancementmeta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedpriority/normalOrdinary priority (the default)source/creatorFiled by the new-issue-creator jobtask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions