Repository navigation
id(ci): nixos-integration VM test still times out silently — #896 acceptance recurrence (nightly 40457980141) #1372
Description
Finding (from #1075 slice 1, queue task 1256)
nixos-integration timed out again in the post-#987/#1041 re-baseline nightly — the same failure mode #896 was closed on.
Evidence: id e2e nightly run 40457980141 on main @ dd4ee06e13393df7d6f2f0ccc72ffd2f5d36e865, concluded failure 2026-10-27T07:51:41Z:
- ❌
checks.x86_64-linux.nixos-integration— builder failed exit code 143; last log lines:timeout reached; test terminating.../Terminated .../nixos-test-driver-id-integration. The server VM booted cleanly (DHCP lease 10.0.2.15, multi-user target reached at ~47s) and then nothing satisfied the test before its timeout (~929s of kernel time in the final log). - In the same run
nixos-serve✅ andnixos-e2e✅ pass — so the id(ci): nightly NixOS VM checks fail — home page no longer contains 'Files' (serve/e2e assertions, integration timeout) #896 home-pageFilesfix held; the integration timeout is the remaining/recurring half of id(ci): nightly NixOS VM checks fail — home page no longer contains 'Files' (serve/e2e assertions, integration timeout) #896, whose acceptance criteria included "nixos-integration no longer times out: it passes, or its failure mode becomes a fast, legible assertion." That criterion is not met on current main. - No commit between
dd4ee06eand currentmain(6f4532ab, +47 commits) touchespkgs/id/nix/tests/*or the id flake, so this is not already fixed upstream.
Acceptance criteria
- Determine what the integration test waits on that never arrives (legible early assertion / logged wait target), instead of a silent ~15-min timeout.
nixos-integrationpasses in a fullnix flake checkrun, or fails fast with a named assertion.- Result recorded on id: validation — Phases 0–8 + platform completion acceptance audit #1075 (slice 1 follow-up).
Filed per #1075's rules: every failure/gap gets its own issue, linked from the audit.
Metadata
Metadata
Assignees
Labels
Package/app: id (pkgs/ or apps/ in this repo)CI/CD pipelines, workflows, and build automationOrg Project gate: active priority — orchestrator adds to the boardImportant and time-sensitiveFiled by the new-issue-creator jobTriaged and routed (see agent note)Queue packet kind: verify, reproduce, and route this item
Activity
developing-today commentedon Oct 27, 2026
CODE-468
dezren39 commentedon Nov 2, 2026
🤖 Agent note · done
cc46ea3f4) already raisedpkgs/id/nix/tests/integration-test.nixtoglobalTimeout 3600. The newest nightly, run 41488645351 (main7297b5d3), fails insidenix flake check (full)before any VM boots: Magic Nix Cache is throttled (HTTP 418, GitHub Actions cache rate-limited; some store paths report no substituter), and the id lib/test targets report E0252/E0255/E0277/E0308/E0597/E0624 plus web-fmt (18 errors) and shell-format drift. Thenixos-test-driver-id-integrationderivation fails only because 2 dependencies failed — it was never evaluated as a timeout.not_before2026-11-02T23:30Z): diagnose the real first failure from run 41488645351, fix compile breakage + formatter drift so the check reaches the VM driver, and separately harden the substituter/cache path. No silent-timeout claim may be re-made from a run that never booted the VM.42 remaining items
developing-today commentedon Nov 3, 2026
agent-note:v1
state: reviewed (coverage loop-back)
Coverage pass 2026-11-03, loop-back review: this issue was re-reviewed (drift / overlap / already-handled / stale / blockers / ordering) so every open issue in the four repos is reviewed and in the system. Verdict: still live; in the system as task 8475 (triage, queued, priority 50) in repo-health-code-pilot. Labels audited and synced. If it is already handled at current main, its triage closes it with evidence; if oversized, the triage files the split; blockers belong on this issue.
Found via: coverage loop-back 2026-11-03
dezren39 commentedon Nov 4, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 4, 2026
agent-note:v1
event: picked_up
task: 8475 (triage, repo-health-code-pilot; repo-worker-code; OpenChamber scheduled lane aea12c82-c5af-4e5a-9b66-f8e360f87177)
session: ses_e5e5b125581ffe5FYJfppHLLtG
state: Claimed for fresh current-main/CI verification. Issue #1372 is OPEN; cited run 40457980141 is from 2026-10-27, while #4913 (legible integration-test failures) and #4696 (pre-VM build fixes) are MERGED. Existing issue note says keep open for one green post-fix nightly after #4850/#4960. Next: verify those prerequisites and latest nightly; no source changes yet.
dezren39 commentedon Nov 4, 2026
agent-note:v1
event: triaged
task: 8475 (triage, repo-health-code-pilot; repo-worker-code; OpenChamber scheduled lane aea12c82-c5af-4e5a-9b66-f8e360f87177)
session: ses_e5e5b125581ffe5FYJfppHLLtG
verdict: STILL LIVE for the post-fix acceptance check; the original silent-timeout evidence is stale and the legibility fix is already merged.
evidence: Fresh main is 71a9ff1 (2026-11-04T05:17:59Z). PR #4913 (named hanging group / legible VM output) is MERGED at a1618c2 with checks SUCCESS; PR #4696's pre-VM build fixes are also merged. Cited run 40457980141 is 2026-10-27 on dd4ee06 and predates these fixes. The latest post-fix nightly, #41662210197 on 443e1d9, was CANCELLED, not green; prior completed run #41488645351 failed before the VM booted. No successful full post-fix nightly is recorded, so do not close #1372 yet.
blockers/order: #4850 remains OPEN (ChainExt test literals/checkpoint compile issue); #4960 remains OPEN with queue/claimed + status/in-progress (integration VM id binary needs ssh feature). Resolve these first, then run a full id e2e nightly on current main and close #1372 only after one green post-fix run. No code changes or new follow-up task from this triage.
dezren39 commentedon Nov 4, 2026
agent-note:v1
event: done (triaged)
task: 8475 (triage, repo-health-code-pilot; repo-worker-code; OpenChamber scheduled lane aea12c82-c5af-4e5a-9b66-f8e360f87177)
session: ses_e5e5b125581ffe5FYJfppHLLtG
state: Completed triage. PR #4913 is merged and the original timeout report is stale; #1372 remains OPEN only for a full green post-fix nightly. Latest post-fix nightly #41662210197 was cancelled, and #4850/#4960 remain open. Findings and ordering are in the triage comment above; no code or follow-up task created.
labels: status/triaged set; queue/claimed, status/in-progress, and work/assigned removed.
bookkeeping: sandbox backlog/sweep log deferred to fleet reconciliation.