design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix

Purpose

Design the egress policy for the basement cluster before anything popular lands on it: how traffic splits between Cloudflare, Turso, Tailscale and the direct home link, with a written budget per traffic class. Child of #1091 (cloud extension — egress half); absorbs #1814, which carried this content but had no home in the program structure.

The constraint: bandwidth is the scarce resource, not compute. The basement fleet has very fast CPUs, ≥3 TB RAM and ~200 TB of disks at near-zero marginal cost — behind exactly 2 Gbps symmetric home fiber.

Source: idea register 2026-10-20 (goal stand-up-lab-cluster-talm-talos-cozystack-postgres-rustfs), plus the superseded #1814 body.

What we know

One fiber provider (Brightspeed) in his part of Osseo: 2 Gbps up/down. A second line = 4 Gbps total — still small at scale. Cable offers 100 Mbps, more expensive (possible redundancy pickup). TriCounty Communications fiber may reach his part of Osseo → then buy both fibers.
Hotspot plans and Starlink investigated for emergency backup; Starlink would likely need a business plan for this use.
Strategy direction (Drew): "tunnel most or much of that to Cloudflare or Turso or anywhere we connect with Tailscale" — shunt bulk traffic to SaaS with cheap or free egress (e.g. Cloudflare R2's zero-egress pricing, CF cache/tunnels), compress aggressively, and only cross the home link with what must.
Budget guardrails: Turso $5 plan and Cloudflare $5 Workers plan in play; free tiers preferred; a $25–$30 Cloudflare plan is possible; $75 is out.

Acceptance

Classify the app's traffic: what can be cached at the edge (CDN), what can live on free-egress object storage (R2), what genuinely must be served from home.
Design the tunnel topology: cloudflared from the cluster, Tailscale links, Turso edge replicas — with per-path egress costs written down.
Compression plan per traffic class (static assets, API payloads, media) and the CPU cost of doing it on the Threadrippers (plenty available).
Failure/redundancy matrix: fiber cut, cable fallback (100 Mbps — what degrades, in what order), Starlink/hotspot emergency tier.
Output: a one-page egress budget model — expected GB/month per class vs the ~2 Gbps link and each SaaS allowance — kept current in the subscription/usage ledger.
Design recorded on this issue or a linked design doc/PR, stating the bandwidth budget against the 2 Gbps symmetric constraint and what happens when it is exceeded.
Follow-up implementation issues filed and linked for any build work the design calls for.

Open questions

Is TriCounty fiber actually available at the address? (Needs an address check.)
Which SaaS egress allowances are we already paying for but not using (ties into the cloud ideation ledger)?

Program rules

Part of the basement cluster program (#1817), project home planning/roadmaps/basement-cluster.md. No HashiCorp anything; everything bootstrappable from config, no hand-built state.

agent-note:v1
state: reviewed (coverage loop-back)
Coverage pass 2026-11-03, loop-back review: this issue was re-reviewed (drift / overlap / already-handled / stale / blockers / ordering) so every open issue in the four repos is reviewed and in the system. Verdict: still live; in the system as task 8629 (triage, queued, priority 40) in repo-health-code-pilot. Labels audited and synced. If it is already handled at current main, its triage closes it with evidence; if oversized, the triage files the split; blockers belong on this issue.
Found via: coverage loop-back 2026-11-03

🤖 Agent note · picked up

Agent: queue worker (repo-worker-code), lane code-e
Session / ID: task 8629
Found via: queue claim
Task: Triage issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 (cluster egress design) at current main: confirm the design is still unstarted, check the parent program (EPIC: Basement Talos/Cozystack cluster program #1817 / epic(cluster): cloud extension (OCI free fleet & egress offload) #1091) state, and route the design work.
Expectation: effort s; done = verdict with fresh evidence + the design deliverable routed as its own task.
Links: task 8629 (queue)

🤖 Agent note · handed off

Agent: fleet monitor (repo-fleet-monitor)
Session / ID: task 8809 · repo-fleet-monitor run 2026-11-03 18:12 CST
Found via: monitor reconciliation
Task: Claim evaporated — task 8809 returned to queued and unclaimed in repo-health-code-pilot after the holding run ended without a terminal verb (checkpoint stage: handed_off). Labels reset to the truth: queue/queued + status/needs-triage. The task cap was also raised this run where it had sat dead-in-queue, so it is claimable again.
Expectation: next claimant picks up from the checkpoint stage named above.

🤖 Agent note · handed off

Agent: queue worker (repo-worker-code), lane K (code-k)
Session / ID: task 8809
Found via: queue claim
Task: Handoff of task 8809 for issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 to the opencode claimant; fresh check confirms issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 OPEN, hard-stop labels: none, labels in claim set.
Expectation: claimant launches bound opencode session and drives task 8809 to terminal state
Links: issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030

🤖 Agent note · handed off

Agent: queue worker (repo-worker-code), lane code-h
Session / ID: task 8809
Found via: queue claim
Task: Design task for issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 — re-verified fresh: issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 OPEN with claim labels already synced (queue/claimed, work/assigned, status/in-progress, no hard-stop labels). It also carries user/review: per protocol that is Muse's review lane, not a stop — the bound session should review the egress design choices first and escalate to user/escalated only for a decision that is genuinely Drew's (e.g. a budget commitment). Execution lane assigned opencode; handing off to the laptop claimant, not working directly. Note: task 8809 is at attempt 6/6.
Expectation: claimant launches the bound opencode session to drive task 8809 to a terminal state.
Links: issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030

🤖 Agent note · attention — protocol gap (missing agent note)

Agent: fleet monitor (repo-fleet-monitor)
Session / ID: cron repo-fleet-monitor run 2026-11-04 04:12 CST
Found via: monitor reconciliation (protocol checks, Duty 4)
Task: flag missing agent note
Expectation: the holding agent posts the required agent-note:v1 at its next transition and the review lane looks at why the protocol was not followed

This item carries work/* / queue/claimed labels but has NO comment containing agent-note:v1 — the holding agent did not leave the required agent note (lifecycle protocol §2/§6). Flagged with user/review + meta/protocol-gap. The gap itself needs a human/Muse review of why the protocol was not followed.

Agent note — picked up (OpenChamber scheduled lane)

Agent: queue worker (repo-worker-code), lane 9aca43b4-5439-4939-9f71-323f5d32521b
Session / ID: queue task 8809 (design), claimed at 2026-11-04T18:25:39Z
Found via: atomic queue claim in repo-health-code-pilot.
Fresh state: issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 is open; impl(cluster): establish measured egress budgets and alerting #5203 is open; no related PR found; hard-stop labels are absent.
Next: resume the existing design, review the requested thresholds, and resolve the external ledger requirement without inventing account or traffic numbers.

Agent note — done (OpenChamber scheduled lane)

Agent: queue worker (repo-worker-code), lane 9aca43b4-5439-4939-9f71-323f5d32521b
Session / ID: queue task 8809, completed at 2026-11-04T18:36:52Z
Found via: atomic queue claim in repo-health-code-pilot.
Completed: kept the egress design and linked implementation follow-up impl(cluster): establish measured egress budgets and alerting #5203; added a dated per-direction capacity and per-class budget model to the subscription tracker at /home/user/hatch-personal/workspace/ai-subscriptions/master-tracker.md.
Review: 70/85/95% stages are reasonable for measured class budgets. For a provider with a hard allowance wall, hold/pause optional sync earlier to cover alert latency and in-flight usage; do not wait for 95% as the first protective action.
Remaining unknowns: no deployed workload means monthly per-class GB stays unknown; current Cloudflare/Turso/Tailscale account usage and TriCounty address availability were not refreshed. They remain named unknowns rather than invented values; impl(cluster): establish measured egress budgets and alerting #5203 tracks measurement and alerts.
No code changes, PR, or CI. Issue design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030 remains open for owner review; queue task 8809 is complete.

Navigation Menu

Repository navigation

design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030

Purpose

What we know

Acceptance

Open questions

Program rules

28 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

design(cluster): egress strategy for 2 Gbps home fiber — budgets, tunnel topology, failure matrix #5030

Description

Purpose

What we know

Acceptance

Open questions

Program rules

Activity

developing-today commented on Nov 3, 2026

dezren39 commented on Nov 3, 2026

28 remaining items

dezren39 commented on Nov 4, 2026

dezren39 commented on Nov 4, 2026

dezren39 commented on Nov 4, 2026

dezren39 commented on Nov 4, 2026

dezren39 commented on Nov 4, 2026

dezren39 commented on Nov 4, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions