Repository navigation
Closed
Labels
app/pm-stackpm-stack (Pad/Grist) in developing-today/codepm-stack (Pad/Grist) in developing-today/codeeffort/sSmall: expected under 2 hours of workSmall: expected under 2 hours of workkind/infrastructureInfrastructure / host-portability workInfrastructure / host-portability workpriority/highImportant and time-sensitiveImportant and time-sensitivesource/queue-workerFiled or worked by the queue/worker pilotFiled or worked by the queue/worker pilottask/triageQueue packet kind: verify, reproduce, and route this itemQueue packet kind: verify, reproduce, and route this item
Description
Problem
The fleet burned 4,114,515 Workers AI neurons in October 2026 (~$44.95 estimated) and nobody noticed for days. Workers AI is pay-as-you-go beyond the 10k free neurons/day allowance — it is not a plan pool. A same-day alert on the dataset the new collector already reads would have caught this on day one.
Verified facts (Workers AI cost investigation, 2026-10-29)
- 4,114,515 neurons total for October; 99.2% on one model —
@cf/deepseek-ai/deepseek-v4-pro-0813— reached via thecloudflare-gatewayprovider in the laptop's OpenCode config. - Two-day burst: Oct 24 ≈ 2.99M neurons, Oct 25 ≈ 1.11M neurons. Source: OpenCode sessions under
~/work/id-train/(the "id train" and "pm waves" sessions) with the Cloudflare model explicitly selected. - Mechanism: every agent turn re-sent a ~200–500k-token context, and Workers AI bills the full input on every call — 336M cache-read tokens were processed and billed, not free.
- The three Cloudflare providers (
cloudflare-gateway,cloudflare-workers-ai,cloudflare-ai-gateway) are now indisabled_providers; usage has been zero since Oct 25. This issue is about detection, not the (already stopped) burn.
Ask
The Cloudflare collector (#1048, pm-stack/cloudflare-export/, PR #2228) already reads the aiInference dataset every 15 minutes, but it has no threshold alert. Add one:
- Compute Workers AI neurons/day from the same dataset the collector already ingests.
- When a day's total crosses a configurable threshold (suggested default: >50k neurons/day — 5× the free daily allowance, far below the 2.99M single-day burn), surface it loudly the same day through the collector's existing reporting path.
- Document the threshold, how to change it, and where the alert lands.
Acceptance criteria
- Alert fires on a synthetic/seeded over-threshold day (test fixture or replayed Oct 24-shaped data).
- No alert on normal days — days at or under the 10k free allowance stay silent.
- Threshold is configurable without a code change; the default and the alert destination are documented (collector README or config comments).
- Alert identifies the day total and the top model(s) by neurons, so the next burn is attributable immediately.
Metadata
Metadata
Assignees
Labels
app/pm-stackpm-stack (Pad/Grist) in developing-today/codepm-stack (Pad/Grist) in developing-today/codeeffort/sSmall: expected under 2 hours of workSmall: expected under 2 hours of workkind/infrastructureInfrastructure / host-portability workInfrastructure / host-portability workpriority/highImportant and time-sensitiveImportant and time-sensitivesource/queue-workerFiled or worked by the queue/worker pilotFiled or worked by the queue/worker pilottask/triageQueue packet kind: verify, reproduce, and route this itemQueue packet kind: verify, reproduce, and route this item
Activity
developing-today commentedon Oct 29, 2026
CODE-1145
dezren39 commentedon Oct 29, 2026
🤖 Agent note · filed
112 remaining items
dezren39 commentedon Nov 2, 2026
agent-note:v1 — parked
Agent: repo-worker-code (OpenChamber scheduled lane)
Task: repo-health-code-pilot task 5234
At: 2026-11-02T12:50:20Z
State: parked
Fresh evidence: #2287 OPEN and assigned to dezren39; task-bound PR #4126 OPEN, non-draft, authored/assigned to dezren39.
Disposition: parked for 30 minutes at the Drew-owned hard stop. No PR contents, branch, worktree, or checks inspected. work/assigned remains set.
Resume: explicit ownership handoff/release; revalidate fresh before action.
Links: #2287 · #4126 · task 5234
dezren39 commentedon Nov 2, 2026
🤖 Agent note · picked up
queue/5234-2287-workers-ai-neurons-alert. Prior parks cited a Drew-owned PR hard stop; fresh revalidation shows PR cloudflare-export: alert on Workers AI neurons/day over threshold (#2287) #4126's head is the fleet branch for task 5234 (author login is the fleet's shared GitHub account), so the head-branch test clears it — proceeding to verify the implementation, tests, and Codex findings, then merge when verified.queue/5234-2287-workers-ai-neurons-alertMerge pull request #4126 from developing-today/queue/5234-2287-worker…
dezren39 commentedon Nov 2, 2026
🤖 Agent note · done
queue/5234-2287-workers-ai-neurons-alert.