Skip to content

Split charge-candidates producer: staged inputs staged on the sandbox, producer runs on the laptop #4641

Closed

Description

@dezren39

Problem

pm-stack/aggregator/scripts/produce-charge-candidates.ts (#1057, #2205) runs from a sandbox checkout of this repo solely because it shells out to the sandbox-only connector CLIs (/opt/hatch/bin/plaid, /opt/hatch/bin/hatch_gws_cli). Drew's 2026-11-02 rule removes ALL repo copies from the sandbox (work runs on the laptop via SSH), and the producer's standing trigger has already lapsed silently once (runs stopped 2026-11-02 09:18Z).

Split (approved by Drew 2026-11-02)

  • Sandbox = thin fetcher, no repo. A standalone script runs the exact CLI calls the producer makes today (one Plaid transactions-get, two Gmail +triage searches, plus +read for message bodies the producer requests) and stages the raw responses as JSON in the hatch-personal-synced hidden-files area: plaid-transactions.json, gmail-triage.json, gmail-bodies.json, charge-inputs-manifest.json (period + fetch times + per-source status).
  • Producer = laptop. The producer gains a staged-input mode (CHARGE_INPUTS_DIR): it reads the staged raw responses instead of shelling out; the manifest's period governs. Gmail bodies it needs but staging lacks are written to gmail-body-requests.json and picked up on a later run once the fetcher stages them (each body fetched exactly once). Live CLI mode stays the default when the env is unset.
  • Producer output (charges-candidates.json) is written directly at the laptop path the fleet-feed collector already reads. The cross-machine output ride and the sandbox clone both go away.

Acceptance

  • Parity: for a fixed historical window, staged-mode output candidates are identical to live-mode output candidates (verified before merge).
  • Aggregator tests/typecheck green in the worktree.
  • After cutover, one full cycle observed: fresh staged inputs -> fresh charges-candidates.json on the laptop -> collector consumes it.
  • Old sandbox runner (~/tools/produce-charge-candidates-scheduled.sh) retired; no sandbox repo copy recreated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    meta/follow-upFollow-up spun out of earlier work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions