Skip to content

mcpx(daemon): full-package test run is killed by SIGTERM — an in-process server's delayed self-signal terminates the test binary #5262

Closed

Description

@dezren39

mcpx(daemon): full-package test run is killed by SIGTERM — an in-process server's delayed self-signal terminates the test binary

Observed (verified this run, laptop, origin/main 1ed5d03)

go test ./internal/daemon/ in pkgs/mcpx fails reproducibly with the test
binary itself dying of SIGTERM:

  • Full suite go test ./...: FAIL internal/daemon 38.699s — output is just
    signal: terminated, no failing test named.
  • Standalone go test -count=1 ./internal/daemon/: same result at 16.6s,
    31.5s, 36.0s and 68.3s across four runs (the variance is the tell: the
    signal lands on whatever test happens to be running).
  • A clean isolated re-run (no other mcpx test suite active on the machine)
    still fails the same way at 36.0s.
  • Individual tests pass alone, e.g. go test -run TestAHiddenToolCannotBeCalled ./internal/daemon/ → PASS in 0.41s (in the -v full-package run that same
    test was the one running when the binary died).

The other red packages in the same full-suite run are already tracked
(internal/settings #5242/#2716, internal/conformance #2716,
internal/defaults #5211); this daemon failure is not covered by any open
issue (searched: "daemon terminated", "signal terminated", "mcpx daemon
test").

Likely mechanism (leads, not proven)

internal/daemon/server.go:1518 handleShutdown — and the restart handler
at :1530 — schedule a delayed self-signal in a goroutine:

go func() {
    time.Sleep(defaults.DaemonRestartSettle)
    p, _ := os.FindProcess(os.Getpid())
    _ = p.Signal(syscall.SIGTERM)
}()

In production the server is the daemon process, so this is its graceful
exit. In tests the server runs in-process inside the test binary: any test
that drives a shutdown/restart request (or leaves one of these goroutines
pending past its own end) SIGTERMs the entire test process
DaemonRestartSettle later, mid-way through an unrelated test. Tests in
this package also register os.Getpid() as a fake daemon PID
(paths_test.go:333,406,428; takeover_test.go:224), so stop/takeover
paths that signal PIDs read from info records are a second suspect route
to the same outcome.

Why it matters

  • The daemon package's tests cannot be trusted as a gate: the package is
    red on main for a reason unrelated to any assertion, and the failure
    report names no test, so real regressions hide inside it.
  • The same in-process self-signal pattern is a hazard for any embedder
    that runs the daemon server inside a longer-lived process.

Acceptance criteria

  • go test -count=1 ./internal/daemon/ passes repeatedly (say 5
    consecutive runs) on a machine with no other mcpx suite running, and
    also when run as part of go test ./....
  • The shutdown/restart self-signal is suppressed or redirected when the
    server runs in-process for tests (e.g. an injectable exit hook, like
    the existing execSelf seam at server.go:1527), without changing the
    production daemon's exit behaviour.
  • A regression test proves that handling a shutdown request in-process
    does not deliver SIGTERM to the host process.

Activity

  1. added
    kind/ciCI/CD pipelines, workflows, and build automation
    task/small-fixQueue packet kind: bounded fix with reproduction and targeted tests
    queue/queuedHas a queued task in the Supabase agent queue
    source/creatorFiled by the new-issue-creator job
    priority/normalOrdinary priority (the default)
    app/mcpxPackage/app: mcpx (pkgs/ or apps/ in this repo)
    on Nov 4, 2026
  2. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · state: filed

    • Agent: new-issue-creator job (new-issue-creator-code-b)
    • Session / ID: cron new-issue-creator-code-b, run 2026-11-03 20:20 CST
    • Found via: filed this run
    • Task: Filed this issue — go test ./internal/daemon/ (pkgs/mcpx) kills its own test binary with SIGTERM on clean main. Verified on the laptop at origin/main 1ed5d03: full suite FAIL 38.7s, four standalone -count=1 runs FAIL at 16.6/31.5/36.0/68.3s (incl. one clean isolated run), output only signal: terminated; individual tests pass alone (TestAHiddenToolCannotBeCalled 0.41s). Lead: internal/daemon/server.go:1518 handleShutdown (and :1530 restart) schedule a delayed os.Getpid() SIGTERM after defaults.DaemonRestartSettle — correct for a real daemon process, fatal when the server runs in-process in the test binary.
    • Expectation: effort s — route the shutdown/restart self-signal through an injectable seam (cf. execSelf, server.go:1527) so in-process servers never SIGTERM their host; done when go test -count=1 ./internal/daemon/ passes 5 consecutive runs and as part of go test ./..., with a regression test that an in-process shutdown request delivers no SIGTERM to the host process.
    • Links: queue packet enqueued to repo-health-code-pilot this run
  3. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 4, 2026
  4. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    task: 9108
    kind: small_fix
    state: picked_up
    lane: code-h (direct execution)
    next: reproduce on the laptop at current main, then route the shutdown/restart self-signal through an injectable seam in pkgs/mcpx/internal/daemon/server.go so an in-process server never SIGTERMs its host; regression test + 5x daemon suite runs.

  5. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    agent-note:v1
    task: 9108
    kind: small_fix
    state: parked (CI pending — runner queue)
    lane: code-h (direct execution)
    done: fix implemented on branch queue/9108-5262-daemon-sigterm (commit 2859c03); PR #5318 open with "Fixes #5262". signalSelf seam routes both self-SIGTERMs (shutdown + restart fallback); production value is byte-for-byte the old behaviour. Regression test TestHandleShutdownSignalsSelfThroughSeam added; restart/handoff tests capture via the seam instead of signal.Notify.
    evidence: go test -count=1 ./internal/daemon/ — 5 consecutive passes on the laptop (62.6/48.7/57.5/66.0/99.9s); go vet + gofmt clean. PR checks (mcpx-fix-carries-a-test, line-limit) still QUEUED after a 5-minute poll — same runner starvation as PR #5277's line-limit run.
    next: on resume, read PR #5318 checks + Codex findings fresh; merge when green, then complete task 9108.

  6. 15 remaining items

  7. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · parked

  8. added and removed on Nov 4, 2026
  9. added
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 4, 2026
  10. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  11. dezren39 commented on Nov 4, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/mcpxPackage/app: mcpx (pkgs/ or apps/ in this repo)kind/ciCI/CD pipelines, workflows, and build automationpriority/normalOrdinary priority (the default)source/creatorFiled by the new-issue-creator jobtask/small-fixQueue packet kind: bounded fix with reproduction and targeted tests

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions