Skip to content

Firefox e2e sandbox hardening: autosave save-count race + websocket-spec server stall (#229 follow-up) #1230

Closed

Description

@dezren39

Follow-up to #229, from queue task 1240's classification of the isolated Firefox dispatch run 40402931464 (PR #987 branch). Nine of the 13 persistent failures are timing/environment artefacts of the nix-VM e2e sandbox, in two clusters:

A. Autosave exact save-count race — autosave.spec.ts:145/:191/:226 fail only on exact request-count assertions (expected 1/2, received 2/3; one retry 3). The tests race the app's 2s debounce autosave against a manual Ctrl+S/click save (test comment: "Hit Ctrl+S immediately (before 2s debounce)"); under VM-Firefox slowness the debounce also fires, the parallel saves hit the server's real 5s per-file save rate limiter (DEFAULT_SAVE_COOLDOWN, pkgs/id/src/web/mod.rs) — the save button visibly entered retry… in :145 attempt 1 — and the 429-retry adds further requests. All saves themselves succeed; no content assertion failed. Open sub-question a local repro can settle: does a manual save fail to cancel a pending debounce (product defect), or is this purely test timing?

B. Websocket-spec server stall window — websocket.spec.ts:488/:519 fail with apiRequestContext.get: socket hang up on the post-save GET /blob/<hash>; :644/:657/:676/:713 all time out in setupCollabPair (page1.fill("#new-file-name") never satisfiable). Timeline: tests 1–12 pass in 2–4s each; one degraded window 23:33:54–23:39:41Z (connection resets → brief recovery, Error Recovery tests pass → ~4 min home page unservable → spontaneous recovery, :761/:779 pass). All pure-websocket behaviour tests passed.

Acceptance criteria

Evidence pack: goal hidden_files/probe-2026-10-27-t1240/ (failures.txt, progress.txt), classification comment on #229 by queue task 1240.

Activity

  1. added
    kind/ciCI/CD pipelines, workflows, and build automation
    source/queue-workerFiled or worked by the queue/worker pilot
    on Oct 27, 2026
  2. developing-today commented on Oct 27, 2026

    @developing-today
    Contributor
  3. added
    priority/normalOrdinary priority (the default)
    queue/queuedHas a queued task in the Supabase agent queue
    area/ciCI/CD pipelines, workflows, and build automation (area)
    app/idPackage/app: id (pkgs/ or apps/ in this repo)
    task/ci-investigationQueue packet kind: investigate a CI failure
    on Oct 27, 2026
  4. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Oct 31, 2026
  5. 104 remaining items

  6. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

  7. added
    effort/mMedium: expected under 1 day of work
    mode/onlineFully doable remotely: design, config, code, cloud
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    and removed
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    on Nov 1, 2026
  8. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    state: triaged (coverage pass)
    Coverage pass 2026-11-03: reviewed in the fleet-wide per-issue pass (drift / overlap / already-handled / stale / blockers / ordering). Verdict: still live; queued in repo-health-code-pilot as triage at priority 40 (priority/normal). If this is already handled at current main, the triage closes it with evidence; if it needs splitting, the triage files the split.
    Found via: coverage pass 2026-11-03

  9. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    state: closed (delivered at main)
    task: 7664 · queue repo-health-code-pilot · worker repo-worker-code · lane bf419306-c4ef-470f-a553-beabf93239de
    verified_at_main: d4d7834 (2026-11-03)

    Both acceptance clusters are satisfied by merged code at current main, so this closes rather than parking.

    Cluster A — autosave exact save-count race: fixed. pkgs/id/e2e/tests/autosave.spec.ts:61-67 now judges counts by quiescence (waiting until the count has been stable longer than the 2s debounce) instead of asserting an exact request count, with the reason documented inline. Merged as PR #3191 (merge a1aa07d8, 2026-11-01). The debounce-coalescing assertion itself is retained, so the coverage did not weaken.

    Cluster B — websocket-spec server stall window: diagnosed and hardened. pkgs/id/e2e/tests/websocket.spec.ts now carries the server-side evidence the acceptance criteria asked for, in-file at :17-56: the upgrade request arrives but the WS handshake never completes after ~54 sequential tests in one Firefox process, no Client connected/Sending Init, HTTP still fine, and a retry on a new socket always succeeds. The hardening is a per-worker fresh browser launch (base.extend browser fixture, :76-86) plus 30s headroom for reconnect cycles, replacing the shared-process flake. Present at main since 2f62ce6c9 (2026-11-02).

    What this does not claim: I did not observe a green e2e run confirming the flake is gone. The id-e2e-nightly run for 2026-11-02 (41488645351, head 7297b5d3) failed before reaching the Playwright specs — nix flake check (full) died on Rust compile errors (id lib, 8 errors) and a just-recipes.json shfmt drift on that stale head, which is unrelated to either cluster and is the reason no e2e result exists for the current code. Reopen this issue if Firefox WS degradation or an autosave count flake recurs on a green run; the two specs above are where to look.

    Side observation for whoever owns CI state: the 11-02 nightly failure is a stale-head build failure, not an e2e failure — worth a separate CI item if it is not already covered by the main-broken tracking.

    Found via: coverage pass 2026-11-03 (queue task 7664)

  10. removed
    task/triageQueue packet kind: verify, reproduce, and route this item
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/idPackage/app: id (pkgs/ or apps/ in this repo)area/ciCI/CD pipelines, workflows, and build automation (area)effort/mMedium: expected under 1 day of workkind/ciCI/CD pipelines, workflows, and build automationmode/onlineFully doable remotely: design, config, code, cloudpriority/normalOrdinary priority (the default)source/queue-workerFiled or worked by the queue/worker pilottask/ci-investigationQueue packet kind: investigate a CI failure

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions