You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
User suggestion from the first full-TCG runs (#76 context): a hung test file currently burns the whole job timeout (300 min on full TCG legs) in silence. The sequential per-file loop in integration.yml should cap each file's runtime so a hang becomes a fast, attributed failure — while staying data-driven, not guess-driven.
Sketch
We already record per-file wall-clock per platform × target in ci-data (runs/*.ndjson, kind:test-file). That gives an empirical baseline per (file, platform).
Wrap each file in the loop with a cap: cap = clamp(factor × observed_max(file, platform), floor, job_remaining) — factor ~3×, floor ~10 min (TCG floor higher, e.g. 20 min), so normal variance never trips it.
Simplest v1: a static per-platform cap baked into the plan matrix (e.g. 15 min accelerated / 45 min TCG per file), revisited against ci-data once more full-TCG runs exist. Data-driven lookup (fetch tested timings in the plan job) is the v2 if static proves too blunt.
On cap hit: kill the bun process and reap QEMU orphans immediately (today the reap step only runs after the whole loop — a starved runner can't even reach it, see CI: macos-x86 (TCG) full suite hangs silently mid provisioning.test.ts #76), write <file> <cap>s timeout to the timing file, continue to the next file. Red job at the end, but every remaining file still reports.
timeout status flows into metrics.ndjson so ci-data distinguishes hang from fail.
Not aggressive by design: the cap exists to convert silent starvation into signal, not to police slow-but-progressing suites (the 300-min job timeout stays the outer bound).
User suggestion from the first full-TCG runs (#76 context): a hung test file currently burns the whole job timeout (300 min on full TCG legs) in silence. The sequential per-file loop in
integration.ymlshould cap each file's runtime so a hang becomes a fast, attributed failure — while staying data-driven, not guess-driven.Sketch
ci-data(runs/*.ndjson,kind:test-file). That gives an empirical baseline per (file, platform).cap = clamp(factor × observed_max(file, platform), floor, job_remaining)— factor ~3×, floor ~10 min (TCG floor higher, e.g. 20 min), so normal variance never trips it.<file> <cap>s timeoutto the timing file, continue to the next file. Red job at the end, but every remaining file still reports.timeoutstatus flows into metrics.ndjson so ci-data distinguishes hang from fail.Not aggressive by design: the cap exists to convert silent starvation into signal, not to police slow-but-progressing suites (the 300-min job timeout stays the outer bound).
🤖 Filed from user direction by Claude Code