Skip to content

mcpx: mcpclient stress suite hits rotating pre-existing DATA RACEs (legacy reinit + handoff tests) #2842

Closed

Description

@dezren39

Found by the queue worker while shepherding PR #2760 (issue #2079) — the stress: PR changed tests (race/shuffle/repeat) job fails with a DIFFERENT pre-existing DATA RACE on successive runs, in legacy-session / handoff code that the PR under test does not touch:

  1. Run 41129819201 attempt 1 (PR feat(mcpx): handle upstream -32042 in CallTool — broker URL elicitations, retry once (#2079) #2760 head f3cd691): TestLegacySession404Recovery/concurrent-calls-share-one-reinitialize — races between mirrorsHeaders() (modern.go:600) / callReissue() (client.go:1068) reads in CallTool and finishLegacy() writes (client.go:460) during concurrent re-initialization.
  2. Same run, rerun attempt: TestClientHandoffCommitsOnlyWhenActivated DATA RACE — also independently observed by another worker lane on unrelated run 41112180105.
  3. Earlier run 41121731190 on the same PR: TestDetachRefusesWhileACallIsInFlight DATA RACE (handoff Detach vs stdio recvLoop).

The same PR head passed the stress job outright on run 41121731190 (mcpx workflow success, head 3adca52), and each rerun fails on a different test — the signature of a flaky race family in the legacy-reinit/handoff surface, not a regression in any one PR.

Acceptance:

  • Reproduce each race locally with go test -race -count=N -shuffle=on on the named tests against clean main.
  • Fix the unsynchronized field access (the finishLegacy writes vs the header-mirror / reissue reads are the concrete pair from run 41129819201) or serialize the concurrent-reinit path.
  • Stress job green on repeated runs for the affected tests.

Filed as the pre-existing-failure record for PR #2760 merging under the standing criteria (its own tests verified; Codex clean on head f3cd691).

Acceptance criteria

  • Each named race (legacy reinit mirrorsHeaders/callReissue vs finishLegacy, TestClientHandoffCommitsOnlyWhenActivated, TestDetachRefusesWhileACallIsInFlight) is reproduced on clean main with go test -race -count=N -shuffle=on, or documented as no longer reproducible with the runs cited
  • The unsynchronized field access in the legacy-reinit/handoff surface is fixed (or the concurrent-reinit path is serialized) with the race tests passing repeatedly under -race
  • The mcpx stress job is green on repeated runs for the affected tests — no rotating DATA RACE failures across reruns

Activity

  1. added
    kind/bugSomething isn't working (prefixed form of the classic bug label)
    app/mcpxPackage/app: mcpx (pkgs/ or apps/ in this repo)
    on Oct 30, 2026
  2. dezren39 commented on Oct 30, 2026

    @dezren39
    MemberAuthor

    Intake review

    Fleet monitor intake pass (protocol §5). Verified live this run: no merged PR explicitly resolves this issue (search hits were number coincidences in unrelated PR bodies), and the problem statement is grounded in the filer's own run evidence.

    Related: Related: PR #2760 / issue #2079 (shepherded when found — the PR itself passed the stress job on one head; the races are in code it does not touch).

    Notes / risks / suggested approach: A rotating race family is worse than a single flaky test: it taxes every mcpx PR's stress job with reruns and can mask a real regression. Risk of the fix itself: medium — concurrency changes in legacy-session/handoff code need the -race suite run repeatedly. Suggested approach: reproduce per-test with -race -count on clean main first, fix the finishLegacy write/read pair, then the handoff pair.

  3. added
    source/queue-workerFiled or worked by the queue/worker pilot
    priority/normalOrdinary priority (the default)
    effort/mMedium: expected under 1 day of work
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    on Oct 30, 2026
  4. dezren39 commented on Oct 31, 2026

    @dezren39
    MemberAuthor

    Observation from queue task 3108 (lane C, 2026-10-31): PR #2975 stress run 41198959299 also failed internal/daemon TestRegistryHandoffKeepsTheChildAcrossTheSuccessor (0.31s) alongside the mcpclient handoff race in this issue. The daemon test passed 5/5 with -race on the PR branch on the laptop, and TestClientHandoffCommitsOnlyWhenActivated reproduced its DATA RACE on clean main d4279a4 (2/3 count=10 race/shuffle runs) as well as on the branch — same pre-existing family, not a PR regression. Adding the daemon registry-handoff test name as another member of this family.

  5. added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 3, 2026
  6. developing-today commented on Nov 3, 2026

    @developing-today
    Contributor

    agent-note:v1
    state: triaged (coverage pass)
    Coverage pass 2026-11-03: reviewed in the fleet-wide per-issue pass (drift / overlap / already-handled / stale / blockers / ordering). Verdict: still live; queued in repo-health-code-pilot as triage at priority 40 (priority/normal). If this is already handled at current main, the triage closes it with evidence; if it needs splitting, the triage files the split.
    Found via: coverage pass 2026-11-03

  7. removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 3, 2026
  8. 38 remaining items

  9. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/claimedQueue worker lifecycle state
    on Nov 3, 2026
  10. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · parked

  11. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    CI backlog tracked in issue #5194 (basic-tier Actions runs queued 30+ min without starting). The park on this issue's PR resumes once those runs start and pass.

  12. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 3, 2026
  13. added a commit that references this issue on Nov 3, 2026
  14. dezren39 commented on Nov 3, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    state: done
    agent: repo-worker-code
    task: 8677
    lane: code-n
    at: 2026-11-03T22:53:30Z

    Resumed task 8677 from its awaiting_ci checkpoint and finished it. Fresh verification: PR #5139 checks all green (basic: mcpx fix carries a test — pass; basic: production line limit (2500) — pass; CodeRabbit skipping/neutral). Codex gate: latest Codex review on head 9026fba reports no findings / no unresolved findings; the earlier P1 (detached-read regression coverage) was dispositioned in that head with TestRecvQueuedBehindDetachDoesNotConsumeSuccessorFrame plus a thread reply.

    Merged PR #5139 (merge commit bc764d4); issue #2842 auto-closed. Completing task 8677 as fixed_merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/mcpxPackage/app: mcpx (pkgs/ or apps/ in this repo)effort/mMedium: expected under 1 day of workkind/bugSomething isn't working (prefixed form of the classic bug label)meta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedpriority/normalOrdinary priority (the default)source/queue-workerFiled or worked by the queue/worker pilottask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions