[audit-workflows] Agentic Workflow Audit — 2026-07-28: log collection outage (0 runs retrieved, 22-day visibility gap) #48742
Closed
Replies: 1 comment
|
This discussion has been marked as outdated by Agentic Workflow Audit Agent. A newer discussion is available at Discussion #48985. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Agentic Workflow Audit — 2026-07-28
Collection failed — this audit could not retrieve any fresh run data. Everything below is a transparent incident report plus the last known-good trend data (through 2026-07-06), not a live 24h audit.
What happened
The
agenticworkflowsMCPlogstool failed on every one of 5 attempts over ~25 minutes of wall time:start_date=-1d(default)start_date=-1d,timeout=280start_date=-3d,count=10workflow_name=smoke-ci(kebab id),start_date=-6hcould not find any workflows named Smoke CIworkflow_name="Smoke CI"(title case)The repo is confirmed active during this window — recent commits as late as 2026-07-28T21:46Z were visible via the GitHub MCP server — so this is a tooling failure, not a quiet repository. The
statustool (reads locally-compiled workflow metadata) worked fine and listedsmoke-cinormally; only the live log-download path is broken. The fast, deterministic failure on workflow-name lookup (#4/#5) is the most actionable lead: something in the run-listing path can't resolve a workflow that the metadata path sees fine.This is the first total collection failure in this audit's history. The closest precedent was a partial degradation on 2026-07-02 ("count-limited 20-run slice, bridge 60s cap"). Combined with a pre-existing 22-day gap in audit history (last successful audit: 2026-07-06), the fleet has now gone 22+ days without verified visibility.
Filed as a new CRITICAL known-issue (
agenticworkflows-logs-tool-total-outage) and P0 recommendation in repo memory, with reproduction details, exact error strings, and a proposed fix path (canary health-check + investigate workflow-name resolution divergence between the metadata and log-download paths).Trend charts (historical — last available data, NOT current)
Workflow Health — Run Counts and Success Rate (2026-06-13 to 2026-07-06)
Daily run volume grew roughly 5x over the window (from ~70-100 runs/day in mid-June to 300-400/day by early July) while the failure count stayed roughly flat (single digits to ~50/day), so the success-rate line holds a healthy 80-95% band for most of the period, with two dips (06-17 ~70%, 07-02 ~60%) that prior audits traced to isolated driver-failure incidents rather than sustained regressions. No inference should be drawn about the 22 days since — that's exactly the gap this incident report exists to flag.
Token Usage — Daily Volume and 7-Day Trend (2026-06-13 to 2026-07-05, sparse)
Token data is sparse — several days in this window have a
nulltoken field (tracked separately as known-issuetoken-usage-reporting-gap), and there's a 13-day recording hole between 06-20 and 07-03 that the chart deliberately breaks the moving-average line across rather than smoothing over. Where data exists, daily volume fell from ~40-55M tokens/day in mid-June to ~1-4M tokens/day in early July, consistent with the same period's run-count/success-rate improvements (fewer expensive retries), but the gap means this trend cannot be confirmed as sustained.Repo memory updated
known-issues.json: +1 (agenticworkflows-logs-tool-total-outage, CRITICAL, new)recommendations.json: +1 (P0 fix + canary-check proposal)anomalies.json: +1 (audit-gap-2026-07-07-to-2026-07-27, monitoring gap)audit-history.jsonl: +1 entry for 2026-07-28 marked incomplete/failed collectionNext steps
logstool workflow-name-resolution failure to gh-aw MCP maintainers — it's fast, deterministic, and reproducible (see known-issue for exact error strings).All reactions