Repository navigation
Egress strategy: 2 Gbps home fiber behind Cloudflare/Turso/Tailscale #1814
Description
Relocated from dezren39/darwin-nix#505 on 2026-10-28 - it was filed in the wrong repo; developing-today/code is its home. Original content follows, verbatim.
Problem
Bandwidth is the scarce resource, not compute. The basement fleet has very fast CPUs, ≥3 TB RAM and ~200 TB of disks at near-zero marginal cost — behind exactly 2 Gbps symmetric of home fiber. Design the egress strategy before anything popular lands on this cluster.
Source: idea register 2026-10-20 (goal stand-up-lab-cluster-talm-talos-cozystack-postgres-rustfs).
What we know
- One fiber provider (Brightspeed) in his part of Osseo: 2 Gbps up/down. A second line = 4 Gbps total — still small at scale. Cable offers 100 Mbps, more expensive (possible redundancy pickup). TriCounty Communications fiber may reach his part of Osseo → then buy both fibers.
- Hotspot plans and Starlink investigated for emergency backup; Starlink would likely need a business plan for this use.
- Strategy direction (Drew): "tunnel most or much of that to Cloudflare or Turso or anywhere we connect with Tailscale" — shunt bulk traffic to SaaS with cheap or free egress (e.g. Cloudflare R2's zero-egress pricing, CF cache/tunnels), compress aggressively, and only cross the home link with what must.
- Budget guardrails: Turso $5 plan and Cloudflare $5 Workers plan in play; free tiers preferred; a $25–$30 Cloudflare plan is possible; $75 is out.
Acceptance / first steps
- Classify the app's traffic: what can be cached at the edge (CDN), what can live on free-egress object storage (R2), what genuinely must be served from home.
- Design the tunnel topology: cloudflared from the cluster, Tailscale links, Turso edge replicas — with per-path egress costs written down.
- Compression plan per traffic class (static assets, API payloads, media) and the CPU cost of doing it on the Threadrippers (plenty available).
- Failure/redundancy matrix: fiber cut, cable fallback (100 Mbps — what degrades, in what order), Starlink/hotspot emergency tier.
- Output: a one-page egress budget model — expected GB/month per class vs the ~2 Gbps link and each SaaS allowance — kept current in the subscription/usage ledger.
Open questions
- Is TriCounty fiber actually available at the address? (Needs an address check.)
- Which SaaS egress allowances are we already paying for but not using (ties into the cloud ideation ledger)?
Acceptance criteria
- An egress design is recorded on this issue (or a linked design doc/PR) covering which traffic classes go via Cloudflare / Turso / Tailscale / direct, and why.
- The design states the bandwidth budget against the 2 Gbps symmetric constraint and what happens when it is exceeded.
- Follow-up implementation issues are filed and linked for any build work the design calls for.
Activity
dezren39 commentedon Oct 28, 2026
Comments carried over from dezren39/darwin-nix#505 (verbatim, in order):
dezren39 - 2026-10-26T21:32:41Z
Intake review
🤖 Fleet monitor intake pass (protocol §5).
Related: the 2026-10-20 lab-cluster idea register (linked in the body); sibling cluster-design issues dezren39/darwin-nix#501–dezren39/darwin-nix#504 filed the same day.
Live check: still open and unlabelled at intake; no merged PR resolves it — this is a design decision owed before anything popular lands on the cluster, per the body.
Notes / risks: bandwidth (2 Gbps symmetric), not compute, is the scarce resource; a second fiber line or TriCounty availability would change the numbers, so the design should state its assumptions explicitly.
Suggested approach: decide the egress split (bulk/SaaS-shunted vs direct) on paper first, with per-class budgets, then file implementation follow-ups.
dezren39 - 2026-10-27T03:26:10Z
Superseded by #1091 (cloud extension epic - egress half)
The basement cluster program was seeded into its standard structure on 2026-10-26 (roadmap + epics + sub-issue tasks in developing-today/code, all in org Project #4 as Area=lab-cluster). Do not start work from this issue - work is tracked at #1091 (cloud extension epic - egress half). Full mapping is on nixdezren39/nix#508. Left open for Drew to close as superseded.
developing-today commentedon Oct 28, 2026
CODE-748
dezren39 commentedon Nov 3, 2026
agent-note:v1
state: triaged (coverage pass)
Coverage pass 2026-11-03: reviewed in the fleet-wide per-issue pass (drift / overlap / already-handled / stale / blockers / ordering). Verdict: still live; queued in repo-health-code-pilot as triage at priority 40 (priority/normal). If this is already handled at current main, the triage closes it with evidence; if it needs splitting, the triage files the split.
Found via: coverage pass 2026-11-03
dezren39 commentedon Nov 3, 2026
🤖 Agent note · done (triage) — closing as superseded
bf419306-c4ef-470f-a553-beabf93239de(OpenChamber scheduled lane), queue task 7739.task/triage status/needs-triage queue/queued, no PR resolving it, no issue-specific branch. Its own 2026-10-27 comment already says "Do not start work from this issue — work is tracked at epic(cluster): cloud extension (OCI free fleet & egress offload) #1091", left open only for a human to close.