Repository navigation
Nix Playwright E2E suite fails after vendoring dependencies #176
Description
Summary
After the test-e2e derivation was made offline-reproducible and Playwright was aligned with the Nix-provided browser, the Firefox suite launches but the Nix check still fails: 113 passed, 17 failed.
Reproduced locally with:
nix build ./pkgs/id#checks.x86_64-linux.test-e2e -L
Failures span autosave, the empty-home state, editor/image features, rename/delete/search, and WebSocket save tests. Examples include:
- Expected saved content but received the autosave
retry…state; some later API reads ended withsocket hang up. - File list/search/rename assertions could not find files created by their tests.
/hash/{hash}redirected to a different test'salt-text-test-…file rather than the expectedredirect-test-…name.
The Playwright config runs one worker against a single ephemeral server. Investigate test-state isolation, repeated-content/hash alias behavior, and server lifecycle under the Nix sandbox. This surfaced once the missing Cargo Git dependencies could be built; PR #175 does not fix these failures yet.
Related follow-up: #175
Activity
dezren39 commentedon Oct 24, 2026
id train triage (2026-10-24): too-big-needs-design — Nix Playwright E2E stabilization after vendoring (113 passed / 17 failed).
dezren39 commentedon Oct 29, 2026
🤖 Agent note · picked up
dezren39 commentedon Oct 29, 2026
🤖 Agent note · done (triage verdict: CONFIRMED — still red at HEAD, narrowed since filing)
test-e2efailed 113 passed / 17 failed under one worker against one shared ephemeral server (autosave retry/socket hang up, file list/search/rename missing files,/hashaliasing another test's file).pkgs/id/flake.nix:405-429now runs one Playwright invocation per spec and reports every failing file; plus editor-features image test fixes (e2e: editor-features test defects — image locator strict-mode + save payloads missing doc_id (#229 follow-up) #1228).id e2e nightlyon main 2026-10-27 (run 40511024415) concluded failure — builder failures (exit codes 1 and 143) in the full flake check with the id-test-e2e derivation among them; the 2026-10-28 nightly (40702501780) was cancelled in the runner stall. No green nightly on main exists since the isolation fix merged. The nightly logs also carry FlakeHub-auth and Magic Nix Cache rate-limit errors, so the follow-up's first step is separating infra noise from real spec failures — the per-spec runner now makes that enumerable.repo-health:v1:fix:developing-today/code:issue:176) — done = fresh per-spec inventory posted here, residual failing specs fixed (or proven infra-only with re-run evidence), test-e2e green.dezren39 commentedon Oct 29, 2026
🤖 Agent note · parked
Progress — inventory run dispatched
Revalidated fresh: #176 is OPEN, no hard-stop labels; the packet's evidence holds — last scheduled nightly on main 2026-10-27 (run 40511024415) failed, 2026-10-28 nightly (40702501780) was cancelled in the runner stall, and no green nightly exists since the per-spec isolation fix (#229/PR #987) merged.
Step 1 of the packet is a fresh per-spec inventory at HEAD. That run takes ~2h in CI (recent nightlies: 1h57m–3h00m), far beyond a worker shift's CI-wait limit, so instead of waiting: I dispatched
id e2e nightlyonmain— run 40850100317 (queued at dispatch) — and parked this task on it. The per-spec runner reports every failing file, which is exactly the inventory the resumed worker needs; infra noise (FlakeHub auth / Magic Nix Cache rate limits seen in earlier logs) must be separated from real spec failures when reading it.37 remaining items
dezren39 commentedon Nov 3, 2026
🤖 Agent note · picked up + fresh triage state (coverage-pass triage, task 7574)
aea12c82-c5af-4e5a-9b66-f8e360f87177queue/claimed,status/in-progress,work/assigned(the stalequeue/blocked/status/blocked/queue/parkedhad no live task behind them and are cleared).user/reviewkept; the attached note at 12:38Z is the review.The previously stated blocker is now stale. At 2026-11-03T14:42Z:
mainis green on nix-ci: 12 consecutivesuccessruns, 13:32Z–14:12Z, latest at HEADe7510b90. The 12:13Z note here said main failed to compile — that was true of the 2026-11-02 tree, not of current main.nix flake check (full)and never produced a per-spec list. Its log is dominated by Magic Nix CacheHTTP error 418 — GitHub Actions Cache throttled, i.e. cache-throughput noise, nottest-e2eassertions.118 passed / 13 failed, all[firefox]: save-path 5, realtime-collab timeout 4, image-rendering 4). The 2026-10-24 filing numbers (113/17) are 10 days old.Action taken (not a code change): dispatched
id e2e nightlyonmainHEADe7510b90→ run 41656241131, queued 14:42Z. That run is the artifact this triage needs: it runs the fullnix flake checkforpkgs/idincludingchecks.x86_64-linux.test-e2eon KVM, which is the only path to a current per-spec list without a local heavy build.Ordering / fix path (scoped, no Drew dependency):
queue/…fix branch per cluster (per-spec isolation runner Stabilize shared-state failures in Firefox E2E CI #229 means each failing spec is independently addressable): save-path/autosave+WebSocket, empty-home state, file rename/delete/search,/hash/{hash}alias.Not blocked on Drew. Pair #1558 is OPEN and genuinely Drew-side (
user/escalated, hostamdgeneration//run/secrets/openclaw); #176 has no host state to activate. Per protocoluser/reviewis mine to evaluate, and this is the evaluation.Parked, not completed: the per-spec list depends on the dispatched run finishing (90-minute timeout). Resume: read run 41656241131; if complete, post the per-spec list and enqueue the cluster fixes; if still running, re-park 30 min.
dezren39 commentedon Nov 3, 2026
dezren39 commentedon Nov 3, 2026
Triage verdict (task 7574, attempt 5) — blocker re-derived at current main
Fresh state (2026-11-03T17:45Z): #176 OPEN, no hard-stop labels. PR #175 MERGED 2026-10-12 and does not fix these failures (per issue body). Main HEAD is now
8e151e89(17:31Z).The attempt-4 artifact is dead. The dispatched
id e2e nightlyrun 41656241131 (HEADe7510b90) was cancelled — superseded by the scheduled run under the workflow'sconcurrency.cancel-in-progress: true. The scheduled run 41662210197 (HEAD443e1d91) then ran 15:18→16:49Z (~91 min) and was itself cancelled at the 90-min job timeout insideRun full nix flake check. Neither produced a per-spec list, and both SHAs are now stale.Current blocker (explicit): no per-spec inventory exists at current HEAD, and the nightly as configured cannot produce one.
nix flake check ./pkgs/id/bundles VM image builds that eat the 90-min budget before/around the e2e specs;cancel-in-progresskills dispatches when the schedule fires; thenix ciqueue is backlogged (many runs queued at 17:3xZ). Last seven nightly runs: 4 cancelled, 3 failed pre-spec (compile/cache). This is a throughput/timeout blocker, not a Drew host action — #176 reproduces from any checkout, no host state to activate (pair #1558 is the genuinely Drew-side item).Best inventory remains 2026-10-29 (118 passed / 13 failed, all
[firefox], posted above): save-path 5 (autosave ×3 + websocket-save ×2), realtime-collab timeouts 4, image-rendering 4. Filing numbers (113/17) are 10 days stale. Coverage gap:tests/world-surface.spec.ts(7th spec file, touched 2026-11-03) postdates that inventory and has never been inventoried.Fix path scoped at current main (no Drew dependency):
nix build ./pkgs/id#checks.x86_64-linux.test-e2e -Lon a KVM-capable host — instead of the fullnix flake check(or raise the nightly timeout past 90 min and dropcancel-in-progress). Per-spec isolation (Stabilize shared-state failures in Firefox E2E CI #229) is already landed:flake.nixloops one Playwright invocation per spec file, each on a fresh ephemeral server.queue/…fix branch each: (a) manual-save trigger path, (b) WebSocket broadcast/subscription convergence, (c) image nodeView render/alt-text/resize. Includeworld-surface.spec.tsin the fresh inventory.Not done here: no code/branch/worktree — laptop SSH was unreachable from this sandbox session (no egress proxy env), and triage needs no code change; no follow-up packet enqueued (a vague packet without the inventory would just re-cycle — re-queue this issue alongside a green-main e2e-capable run per the 12:13Z note).