Repository navigation
ci: no PR check evaluates a NixOS host — PR #699's broken amd service merged green and failed only on the laptop switch #705
Description
Finding
No PR check evaluates any NixOS host configuration, so a host definition that fails at evaluation merges green and is only discovered at just switch time on the machine itself. This already happened:
-
PR amd: KRdp remote desktop, declaratively (package + user service) #699 (commit
7beeff8b) addednixos/hosts/amd-krdp.nixwithsystemd.user.services.app-org.kde.krdpserverunquoted. Unquoted, the dotted name parses as nested attrsets, and the value violates the systemd leaf-value type. -
PR amd: KRdp remote desktop, declaratively (package + user service) #699's checks were all green — verified via its commit's check-runs:
runsuccess,Nix flake evaluationsuccess,treefmt checksuccess. -
PR fix(amd): quote KRdp service name so it is one systemd unit #701's body records the first switch attempt on amd failing during evaluation:
error: A definition for option `home-manager.users.user.systemd.user.services.app-org.kde.krdpserver' is not of type `boolean or signed integer or string or absolute path or list of (...)'PR fix(amd): quote KRdp service name so it is one systemd unit #701 (commit
bfe44fe6) fixed it by quoting the name — a second PR and a failed switch for a one-character-class bug that any host evaluation in CI would have caught pre-merge. PR amd: KRdp remote desktop, declaratively (package + user service) #699's own body even claimed a targeted eval had verified the service, so the manual eval is demonstrably not a reliable net here.
Evidence (current main f7dfea35)
-
.github/workflows/nix-ci.yml:60-70— theflake-checkjob named "Nix flake evaluation" (the PR gate that passed on amd: KRdp remote desktop, declaratively (package + user service) #699/fix(amd): quote KRdp service name so it is one systemd unit #701) runs exactly three evals, none of them a host:nix eval .#devShells.x86_64-linux.default.drvPath nix eval .#packages.x86_64-linux.mcpx.drvPath nix eval .#apps.x86_64-linux.format-check.program(Verified two ways: local clone
git show origin/main:.github/workflows/nix-ci.ymlandgh api repos/developing-today/code/contents/.github/workflows/nix-ci.yml— lines 68-70 agree.) -
A grep of
.github/workflows/fornixosConfigurationsfinds exactly one hit:jules-daily-flake-sync.yml:71, which evaluates.#nixosConfigurations.nixos.config.system.build.toplevel.drvPath(the Intel host, not amd) inside a scheduled Jules prompt — not a PR gate. -
Prior instance of the same discovery pattern: amd host fails at evaluation: no outputHashes for p2panda-core-0.7.1 #158 (closed) — "amd host fails at evaluation" — was also found manually, not by CI.
Single-repo: the NixOS hosts (nixos/hosts/, amd, nixos) live only in this repo; dezren39/nix has no nixos/ host tree and no equivalent workflow.
Why it matters
The hosts are Drew's actual machines. An evaluation error that reaches main surfaces at the worst possible moment — mid-switch on the laptop — instead of on the PR that introduced it. Host modules are edited regularly (KRdp is one example), and the type system only protects them if something actually forces the evaluation.
Acceptance criteria
- A PR check evaluates at least
.#nixosConfigurations.amd.config.system.build.toplevel.drvPath(and ideally thenixoshost's too) — evaluating the drvPath only, no build — or an equivalently deep forced evaluation of the hosts' home-manager/systemd options. - The check demonstrably fails on PR amd: KRdp remote desktop, declaratively (package + user service) #699's original tree (i.e. with PR fix(amd): quote KRdp service name so it is one systemd unit #701's quoting reverted) and passes on current main.
- The check is bounded so it does not blow up PR CI time (drvPath-only eval, a timeout, and/or path filtering to host-affecting changes are all acceptable ways to achieve this — the failure-on-amd: KRdp remote desktop, declaratively (package + user service) #699 demonstration is the requirement, not the mechanism).
Filed by the pr-diff-review job from its review of PRs #699/#701.
Activity
dezren39 commentedon Oct 26, 2026
Intake review
Still live (verified this run, fleet monitor): current main's
.github/workflows/nix-ci.yml(lines 68–70) still evaluates exactly three things —devShells.x86_64-linux.default,packages.x86_64-linux.mcpx,apps.x86_64-linux.format-check— and no PR workflow evaluates anynixosConfigurationshost. The gap PR #699 slipped through is unchanged.Related: PR #699 (the broken amd service, merged green), PR #701 (the fix, with the switch-time error in its body), closed #158 (the same discovery pattern found manually).
Notes / risks: host evaluation is heavier than the current three evals — scope the gate deliberately (eval-only
drvPathof each host's toplevel, as the Jules scheduled workflow already does for the Intel host atjules-daily-flake-sync.yml:71, is the cheap shape). Watch CI time; evaluating both hosts on every PR may warrant path filtering tonixos/**+ flake files.Suggested approach: add a PR job/step evaluating
nixosConfigurations.{nixos,amd}.config.system.build.toplevel.drvPath(eval only, no build), triggered at least when host/flake paths change.Labels normalized by the fleet monitor: kind/ci, status/needs-triage, priority/high (the hosts are Drew's actual machines; failure surfaces mid-switch), effort/m, area/ci.
dezren39 commentedon Oct 30, 2026
🤖 Agent note · picked up
134 remaining items
dezren39 commentedon Oct 31, 2026
🤖 Agent note · parked
dezren39 commentedon Oct 31, 2026
agent-note:v1 — Agent: repo-worker-code (lane B); state: claimed. Task 4048 resumed: GREEN direction now verified in CI — run 41231411994 (nix ci @ 25154dc) completed SUCCESS including job "basic: NixOS host evaluation". Codex summary on 25154dc: completed, no findings; 0 unresolved threads. Remaining gate: RED direction — dispatch run 41235601344 on scratch branch queue/4048-705-redcheck is still QUEUED; verifying the red tree on the laptop in parallel (nix eval drvPath must fail with #701 quoting reverted).
Merge pull request #3025 from developing-today/queue/4048-705-host-eval
dezren39 commentedon Oct 31, 2026
agent-note:v1 — Agent: repo-worker-code (lane B); state: done (fixed_merged). Task 4048 completed: PR #3025 merged (568c7b9) — the "basic: NixOS host evaluation" job is now on main in .github/workflows/nix-ci.yml. Verification both directions: GREEN — CI run 41231411994 (nix ci @ 25154dc) completed SUCCESS including the host-evaluation job (plus flake evaluation and proxy fallbacks). RED — the job command (NIXPKGS_ALLOW_UNFREE=1 nix eval --impure .#nixosConfigurations.amd.config.system.build.toplevel.drvPath) run on the laptop against the scratch redcheck tree (queue/4048-705-redcheck @ 4dc37b6, #701 KRdp quoting reverted) exits 1 with the exact #699-class TypeError in nixos/hosts/amd-krdp.nix (option home-manager.users.user.systemd.user.services.app-org.kde.krdpserver). The CI dispatch for the red tree (run 41235601344) was still queued in the fleet runner backlog at merge time; its eventual red verdict corroborates the laptop proof. Codex on 25154dc: completed, no findings, 0 unresolved threads. Scratch branch deleted after merge.