Continuous testnet recorder #7

Why

Stellar testnet resets roughly quarterly and public RPC keeps only a short history. Nobody archives it. A recorder that writes every ledger's operations and events to object storage makes HackProof the surviving record of testnet, which every future hackathon page depends on.

In this repo today

indexer/sync.py pulls the window in 400 ledger chunks from Soroban RPC and Horizon, parses transactions with indexer/parser.py into Event models (indexer/events.py) and writes Postgres. A window older than the RPC retention cannot be synced again.

What to build

• apps/api/src/hackproof/recorder/: a long running job (or hourly cron) that keeps a cursor in object storage, pulls ledgers, transactions (including failed), operations and events from Horizon and RPC since the cursor, and writes gzip jsonl partitioned by date and hour: testnet/YYYY/MM/DD/HH/{ledgers,transactions,operations,events}.jsonl.gz.
• An S3 compatible client (Cloudflare R2 or Backblaze B2). Settings: archive_bucket, archive_endpoint, archive_key, archive_secret.
• Idempotent: rerunning an hour overwrites the same keys. A manifest per day lists the hours present and record counts.
• recorder/read.py with iter_events(start, end) that yields the same Event models as the live parser, so sync can use the archive when the window is older than RPC retention.
• Reset detection: if the latest ledger sequence is lower than the cursor, log a testnet reset, write a resets.json entry with the timestamp and start a new cursor. Never delete old partitions.
• A recorder command in the Dockerfile and a compose service. Document the expected storage per month after a week of running.

Acceptance criteria

• Runs for 48 hours on a $5 VPS without manual intervention, with a log line per hour.
• sync can rebuild the hackmeridian-2026 hourly totals (/api/timeline) from the archive alone and match the live numbers within 1%.
• Reset detection is tested with a fixture where the sequence goes backwards.

See CONTRIBUTING.md for conventions.