Skip to content

mcpx: the darwin skip override in dezren39/nix keys on this repo's check-phase line with a silent replaceStrings — no automated guard against drift #418

Closed

Description

@dezren39

Finding

PR #417 changed the canonical mcpx check-phase line in this repo from go test $pkgs to go test -timeout 30m $pkgs, and correctly updated this repo's own guard. But the same exact string is also the search pattern for the darwin test-skip override in dezren39/nix, where it is applied with builtins.replaceStrings — which silently returns its input unchanged when the pattern does not match. Nothing automated in either repo fails if the two drift apart again; the downstream retarget in dezren39/nix PR #513 was verified only by a manual nix eval noted in that PR's body.

This is one cross-repo finding (one issue, per the set rule). Affected repos:

  • developing-today/code (this repo, PR reviewed): pkgs/mcpx/package.nix:122 now runs go test -timeout 30m $pkgs (main 86a2aa9e648e530e0fdeadc4566e5dd28256ae00, PR mcpx: 30m per-package timeout in the nix check phase (for dezren39/nix#510) #417, commits eaa2018d550e5ef6cc39064f1cac02e869811a25 + f27752c539f338719a902b3e96da439f52637262). Its guard pkgs/mcpx/internal/buildcheck/checkphase_test.go:86-87 requires that exact line — so this repo is protected against its own line changing unnoticed, but nothing signals the downstream consumer.
  • dezren39/nix: flake.nix:317-320 (main b70c39a95ac96ee80c1769dd8e674192ca1c76cf) rewrites the darwin checkPhase with builtins.replaceStrings [ "go test -timeout 30m $pkgs" ] [ "go test -timeout 30m -skip='${mcpxDarwinSkip}' $pkgs" ]. The skip list at flake.nix:323 names 21 takeover/handoff tests that fail on darwin.

Evidence

  • builtins.replaceStrings is a silent no-op on a missing pattern — verified directly: nix eval --expr 'builtins.replaceStrings ["go test -timeout 99m $pkgs"] ["REPLACED"] "go test -timeout 30m $pkgs"' returns "go test -timeout 30m $pkgs" unchanged, no error.
  • The override block itself shows the safer pattern exists a few lines above: the darwin postPatch at dezren39/nix flake.nix:310-311 uses substituteInPlace --replace-fail, which fails the build loudly when its pattern is absent. Only the checkPhase rewrite lacks that property.
  • No automated guard exists today: dezren39/nix's flake checks are formatting, tidy, flake-tidy-tests, plugins-tests, opencode-db-pids-tests, skill-tests (plus darwin-only sidepulse checks) — none asserts the darwin mcpx checkPhase contains -skip=; its mcpx-nixbuild workflow builds only .#packages.x86_64-linux.mcpx, where the darwin override is not applied. This repo's buildcheck guard reads only this repo's package.nix.
  • PR mcpx: 30m per-package timeout in the nix check phase (for dezren39/nix#510) #417's own body named this coupling as a manual follow-up ("its darwin -skip replaceStrings pattern updated to the new canonical line"), and nix PR lab-ops-hub: both crawl validators accept impossible DNS names — labels over 63 chars and names over 253 chars #513 discharged it by hand-checking nix eval .#packages.aarch64-darwin.mcpx.checkPhase. The next canonical-line change (a different timeout, an added flag) will need the same manual catch, with no failure if it is missed.

Why it matters

If the pattern drifts, a routine mcpx-src pin bump in dezren39/nix merges with Linux CI green while the darwin override silently does nothing: the 21 skipped takeover/handoff tests run on darwin and fail there (the flake.nix comment at dezren39/nix:296-302 records that these tests cannot pass on darwin), so the darwin mcpx build breaks — first surfacing on a Mac rebuild, with the skip list still visibly present in flake.nix, which makes the cause non-obvious. This is distinct from dezren39/darwin-nix#471, which tracks a one-time manual aarch64-darwin build verification at the current pin; that build passes today precisely because the pattern currently matches, and it would not catch the next drift.

Acceptance criteria

  • An automated check (in either repo, covering the set) fails when the darwin checkPhase in dezren39/nix does not contain the -skip= rewrite — e.g. a flake check or CI step that evaluates .#packages.aarch64-darwin.mcpx.checkPhase and asserts the skip is present, or a fail-on-mismatch construction replacing the bare replaceStrings.
  • The check demonstrably ablates: with the search pattern reverted to the pre-mcpx: 30m per-package timeout in the nix check phase (for dezren39/nix#510) #417 line (go test $pkgs) it fails, and on the current mains of both repos it passes.
  • The check runs automatically on changes that can break it (a canonical-line change in this repo's pkgs/mcpx/package.nix, or an mcpx-src pin bump / flake.nix change in dezren39/nix), not only as a manual step recorded in a PR body.

Found by the pr-diff-review job while reviewing #417.

Activity

  1. dezren39 commented on Oct 20, 2026

    @dezren39
    MemberAuthor

    Set complete — both halves merged in one sweep pass:

    Ablation (acceptance criterion 2): with the search pattern reverted to the pre-#417 line (go test $pkgs), nix eval .#packages.aarch64-darwin.mcpx.checkPhase fails with the throw (exit 1); on current mains it returns the phase with the full -skip list, and nix build .#checks.x86_64-linux.mcpx-darwin-skip passes. All CI green on both PRs.

  2. added a commit that references this issue on Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions