Repository navigation
design(modal): best use of Modal credits — $30/mo recurring + $100 one-time grant, for Atlas embedding and GPU batch #6069
Description
Design (2026-11-04, Drew's directive): best use of our Modal balance — $30/mo Starter credits (recurring) + $100 one-time hackathon grant = $130 today. Full design doc: ~/workspace/scratch/modal-gpu-design-2026-11-04/design.md (sandbox scratch; move into repo docs when implemented).
Verified facts
- Modal Starter: $30/mo included compute, 10 GPU concurrency, per-second billing. L4 = $0.80/hr, A10 = $1.10/hr, H100 = $3.95/hr. Granted credits expire 12 months after grant → the $100 is spend-by ~Oct 2027.
- Access already exists and is proven:
~/git/fleet/modal-transcribe/(Whisper on L4, 2026-10-31). No new account/credential. - Production Atlas model is arctic-embed-m-v2.0 (768-dim, swap completed 2026-11-03) → Modal output is dimension-identical to the live index, no migration.
Headline math (assumptions labeled in the doc; validate with a ~$1 benchmark first)
- Throughput planning numbers for arctic-embed-m-v2.0 on one L4: 15k (pess) / 30k (expected) / 60k (opt) tok/s. Fleet local baseline is a measured 1,529 tok/s.
- Full Atlas internal re-embed (55.05M tokens, measured): ~31–61 min on L4 ≈ $0.60–$1.15 all-in — vs 10.0 h locally (~20× faster).
- $30/mo ≈ 2.9B tokens ≈ 5.4M chunks ≈ 8–9 GB of source text (L4, expected rate, after container overhead).
- Embed-everything-we-own (all repos, ~464 MB tracked measured, ~100M tokens after filtering): ~$1–2 one-shot.
Recommended allocation
- Recurring $30: weekly delta embed window (<$0.15/run) → monthly bulk/experiment window (chunking experiments and bake-offs become ~$1 passes instead of 10-h local jobs) → monthly feedback-normalization + golden-set eval batch → transcription on demand → ~$10/mo reserve as burst fund.
- One-time $100 (one-shot artifacts): embed-everything first pass (
$2) → finish the Qwen3-0.6B bake-off that died locally at 37,888/101,471 chunks after$3) → comments-corpus embed when the license gate clears (~$5–25) → ~$50 reserve.46 h ($2, closes a verdict hanging since Oct 31) → arctic-embed-l-v2.0 1024-d shadow index + parity eval ( - Ledger: track
modalas a pool in quota-check (credits granted/used, tokens embedded, reuse ratio), alert on projected PAYG overage and on grant expiry <60 days with >$25 unspent. Our scheduler triggersmodal run; Modal's own cron is backup only.
Hard guardrails
- Secret-tier text NEVER goes to Modal (external processor; classification violation regardless of price). Internal tier only. Secret tier stays on the local embedder (638 chunks — laptop rate is fine).
- No OCI anything (stood down 2026-10-31). No Mac in any option (Vertex machine). No always-on GPU. No new vendor accounts.
GPU-after-credits survey (Blacksmith / GitHub Actions GPU / Copilot / OCI / local) is filed separately and linked below — headline: Modal PAYG overage is the next step; Blacksmith has no GPU product; GitHub's native GPU runner is a $3.12/hr T4; we own no discrete GPU (laptop is a Radeon 780M iGPU; the "RTX 3060 Ti" in our records is a Marketplace listing we passed on).
Cross-links: #6018 (Atlas ingestion program), #6019 (ingestion ops + cost — this is its burst-compute answer), #5733 (Jaccard third signal — Modal feeds the vector leg the rescore combines).
Acceptance criteria
- The chosen use of the Modal balance ($30/mo recurring + $100 one-time grant) is recorded on this issue, per the design doc.
- Implementation is filed as linked follow-up issue(s) or landed, with Atlas embedding / GPU batch as the first workload if the design holds.
- Spend is tracked against the $130 total, and the ~Oct 2027 expiry of the one-time grant is respected in the plan.
Metadata
Metadata
Assignees
Labels
Infrastructure, deployment, and operationsMedium: expected under 1 day of workA design proposal (design-architect): multi-step, phased, decision-bearingIntake review done: acceptance criteria, links, and normalized labels addedOrdinary priority (the default)Filed directly by a humanTriaged and routed (see agent note)Queue packet kind: verify, reproduce, and route this item
Activity
developing-today commentedon Nov 5, 2026
CODE-1732
dezren39 commentedon Nov 5, 2026
GPU-after-credits survey filed as #6072 (Blacksmith / GitHub Actions GPU / Copilot / OCI / local inventory, with the post-Modal ladder).
dezren39 commentedon Nov 5, 2026
Intake review
Related: design doc at
~/workspace/scratch/modal-gpu-design-2026-11-04/design.md(named in the body; the body asks for it to move into repo docs when implemented). No merged PR referencing #6069 was found in a title/body search this run.Notes: access is already proven (
~/git/fleet/modal-transcribe/), so this is a prioritisation design, not an access problem. Risk: recurring-vs-one-time credit mix — the acceptance criteria pin spend tracking so the expiring grant is used first.— Fleet monitor intake (repo-fleet-monitor), 2026-11-05. Merged-PR title/body search count for this issue number at intake: 0.
dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
developing-today commentedon Nov 5, 2026
Pointer from triage of issue #6072 (task 10435, lane code-u): the GPU-capacity research feeding this design's §4 is complete, and its single recorded recommendation now stands on issue #6072 — default to Modal PAYG overage when credits run out; GitHub-hosted
gpu-t4-4-corelarger runner as break-glass only; OCI remains stood down under the 2026-10-31 hold. Details and per-question evidence: #60725 remaining items
dezren39 commentedon Nov 5, 2026
🤖 Agent note · picked up
dezren39 commentedon Nov 5, 2026
Triage findings — current main \
Issue #6069 remains OPEN and has no matching PR. Its body already records the selected mix (0/month recurring Starter credits + 00 one-time grant), the 30 spend/expiry controls to implement, and the hard boundary that secret-tier text stays local. The completed research on #6072 records Modal PAYG as the default after credits, with GitHub's \ runner as break-glass only; OCI remains stood down.
Ordering and gates: #6025 is the first Atlas workload and remains OPEN; its triage records gates at #6019 (operations design), #6021 (classification scan; currently queue/queued), and #6028 (delta semantics, or its hash-skip fallback). The separate ledger issue #6058 is OPEN/status-triaged and depends on #6028 semantics. Keep those prerequisites ahead of the first batch run; then implement the ledger against the settled delta behavior. The existing issue split is sufficient, so this triage creates no duplicate follow-up task. Keep #6069 open as the parent until the implementation and spend/expiry tracking are materialized; the design source named in its body is currently in scratch and should move into repo documentation with that work.
🤖 Agent note · handed off
dezren39 commentedon Nov 5, 2026
Correction to my preceding comment: shell interpolation stripped the dollar amounts. The intended allocation is $30/month recurring Starter credits + $100 one-time grant = $130 total; the findings and ordering below are unchanged.
Triage findings — current main 2801e36
Issue #6069 remains OPEN and has no matching PR. Its body already records the selected mix ($30/month recurring Starter credits + $100 one-time grant), the $130 spend/expiry controls to implement, and the hard boundary that secret-tier text stays local. The completed research on #6072 records Modal PAYG as the default after credits, with GitHub's gpu-t4-4-core runner as break-glass only; OCI remains stood down.
Ordering and gates: #6025 is the first Atlas workload and remains OPEN; its triage records gates at #6019 (operations design), #6021 (classification scan; currently queue/queued), and #6028 (delta semantics, or its hash-skip fallback). The separate ledger issue #6058 is OPEN/status-triaged and depends on #6028 semantics. Keep those prerequisites ahead of the first batch run; then implement the ledger against the settled delta behavior. The existing issue split is sufficient, so this triage creates no duplicate follow-up task. Keep #6069 open as the parent until the implementation and spend/expiry tracking are materialized; the design source named in its body is currently in scratch and should move into repo documentation with that work.
🤖 Agent note · handed off