Skip to content

[aw-failures] [aw-fix] P0: Daily Spending Forecast — gh-aw forecast --days 30 times out after 45m during artifact download #48715

Description

@github-actions

Problem

Fix (or bound) the gh-aw forecast --days 30 --json artifact download — it burned the entire 45-minute job timeout and produced zero output, wasting a full run of compute/API budget on a daily scheduled workflow that will fail identically every day until fixed.

Affected workflow / run

  • Workflow: .github/workflows/daily-spending-forecast.lock.yml ("Daily Spending Forecast")
  • Failing job: agent → step "Execute GitHub Copilot CLI"
  • Run: §30384729502 — created 2026-07-28T17:50:38Z, failed at 18:40 with ##[error]The action 'Execute GitHub Copilot CLI' has timed out after 45 minutes.
  • Comparator (nearest successful run of same workflow): §30370544606 — 2026-07-28T14:52:39Z, succeeded

Evidence

The agent launched nohup ./gh-aw forecast --repo github/gh-aw --days 30 --json > forecast.json 2> forecast.stderr2.log & and then polled it every 5 minutes:

  • ~10 min in: 216 artifact downloads
  • ~15 min in: +103 downloads/5min (steady pace)
  • ~20 min in: agent itself estimates "~30-40 min total" needed
  • 45 min in: hard timeout, job killed, no output produced

audit-diff between the successful baseline run and the failed run shows the failed run made only 1 GitHub core API call before timing out vs. 2103 for the successful run — consistent with the process getting stuck early in a slow, serial artifact-download loop rather than making steady API progress. This is a --days 30 window pulling an increasing number of historical workflow-run usage artifacts as the repo's run volume grows; the download step is not scaling.

Probable root cause

gh-aw forecast's usage-artifact download path is serial/sequential (or otherwise not parallelized) and its per-artifact overhead no longer fits a 30-day window inside the job's 45-minute budget as the number of workflow runs in github/gh-aw has grown. This is a scaling regression, not a one-off flake — the linear "~100 downloads/5min" pace observed means any repo with enough daily runs will eventually cross the 45-minute boundary for a 30-day forecast.

Proposed remediation

  1. Parallelize/batch the artifact download in gh-aw forecast (concurrent downloads instead of one-at-a-time), or cache previously-downloaded usage artifacts across runs so each run only fetches deltas.
  2. Add a --days sanity check or progress-based early-exit/streaming output so partial results are usable even if the full window can't complete in time.
  3. Either raise the job timeout for this specific workflow step or reduce the default --days window so it reliably completes well under 45 minutes with headroom.
  4. Add an internal timeout/progress log inside gh-aw forecast itself (not just relying on the outer CI step timeout) so a stuck run fails fast with a clear error instead of silently running out the clock.

Success criteria

Generated by 🔍 [aw] Failure Investigator (6h) · age00 · 140.3 AIC · ⌖ 26 AIC · ⊞ 5.2K ·

  • expires on Aug 4, 2026, 11:23 AM UTC-08:00

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions