Skip to content

Fold script-installed laptop user units into the flake (github-backup, worktree-reaper, quota exports, dashboard-backup, oc-password-sync) #3720

Closed

Description

@developing-today

Problem

On the laptop, most user units resolve into /nix/store — they are home-manager modules in this flake (nixos/home/*, enabled from nixos/hosts/default.nix) and deploy with nixos-rebuild switch. A set of newer systems was instead installed as plain unit files in ~/.config/systemd/user/ by install scripts. They run fine, but:

  • a machine rebuild / DR restore from the flake alone would not bring them back;
  • unit changes bypass the flake's review flow — drift between "what the repo says runs" and "what runs" is invisible;
  • there are two sources of truth for the laptop's service set.

Drew's standing rule (code#1723) is that all laptop services are made by this flake; the script-installed units are the remaining exceptions. Ordered folded in 2026-11-01 ("fix all now").

Scope (verified live 2026-11-01: plain files in ~/.config/systemd/user/, not store symlinks)

  • github-backup-drain / github-backup-fetch / github-backup-reconcile (.service + .timer each)
  • worktree-reaper (.service + .timer)
  • cloudflare-quota-export, meta-quota-export, dashboard-backup (.service + .timer each)
  • oc-password-sync (.service)

Not in scope: rustdesk.service (vendor unit), git-maintenance@ (git's own), oc-sumauto (transient unit, no file exists).

Plan

  • One home-manager module per system under nixos/home/, enabled from nixos/hosts/default.nix next to the existing laptop modules, replicating the current units' semantics exactly (ExecStart, WorkingDirectory, timer schedules, timeouts, Nice). Executed scripts stay at their checkout paths, per the fleet-feed-collector precedent — only unit provenance changes.
  • Equivalence is proven before activation: build the amd configuration and diff the generated units against the installed files field-by-field; any intentional difference is called out in the PR.
  • Activation is careful and reversible: the plain unit files are first moved (not deleted) to ~/.local/state/flake-fold-backup-2026-11-01/ on the laptop, then the established switch path runs; each migrated unit is verified (resolves into /nix/store, timer armed, one functional run per system). Rollback = restore the plain files (+ previous generation) if any check fails.

Dependency

worktree-reaper's ExecStart currently points at the bootstrap worktree ~/git/code-wt-reaper (PR #3698). The fold points it at the canonical primary path ~/git/code/infra/worktree-reaper/... and retires the bootstrap install per that PR's README — so this lands after #3698 merges.

Acceptance criteria

  • Every unit listed in Scope (github-backup drain/fetch/reconcile, worktree-reaper, cloudflare-quota-export, meta-quota-export, dashboard-backup, oc-password-sync) is defined by a home-manager module in this flake under nixos/home/, enabled from nixos/hosts/default.nix.
  • The generated units are diffed against the currently installed files before activation and match on ExecStart, WorkingDirectory, timer schedules, timeouts, and Nice (or every difference is named and justified on this issue).
  • After a rebuild, the script-installed plain files in ~/.config/systemd/user/ are removed or superseded so the flake is the single source of truth, and the services still run.
  • Out of scope units (rustdesk.service, git-maintenance@) are untouched.

Activity

  1. developing-today commented on Nov 1, 2026

    @developing-today
    ContributorAuthor

    Implementation: PR #3722 (branch infra/flake-fold-units, commit 78b4fc0) — six home-manager modules under nixos/home/ plus enablement blocks in nixos/hosts/default.nix.

    Equivalence (proven before activation): the amd configuration was evaluated on the laptop and every one of the 15 generated unit definitions was diffed field-by-field against the installed plain files — 14/15 identical in all fields; worktree-reaper.service differs only in the intentional ExecStart move (bootstrap worktree ~/git/code-wt-reaper → primary ~/git/code).

    Activation is staged, not executed: the switch needs Drew's interactive sudo password (verified sudo -n fails on the laptop), so per the plan the plain unit files are left untouched for now. The single remaining step — move the 15 files to ~/.local/state/flake-fold-backup-2026-11-01/ (dir + runbook already prepared there) and run ~/run-switch.sh — is documented in that directory's README on the laptop.

    Merge order matters: #3698 first (it creates the reaper script's canonical path), then #3722.

  2. added
    kind/infrastructureInfrastructure / host-portability work
    priority/highImportant and time-sensitive
    effort/lLarge: multi-day or open-ended; expect staged check-ins
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    on Nov 1, 2026
  3. dezren39 commented on Nov 1, 2026

    @dezren39
    Member

    Intake review

    Related: code#1723 (standing rule cited in the issue: all laptop services are made by this flake). No merged PR referencing #3720 was found at intake.

    Notes: Filed today with no labels at all. The scope was verified live by the filer (plain files, not store symlinks), and Drew ordered the fold-in on 2026-11-01, hence priority/high. The equivalence diff before activation is the safety-critical step — a unit that differs in schedule or timeout would silently change backup/quota behaviour.

    Risks: Timer or environment drift between the installed files and the module versions; a rebuild that drops a unit before its replacement is proven.

    Suggested approach: One module per system next to the existing laptop modules, executed scripts staying at their checkout paths (fleet-feed-collector precedent in the issue), build the amd configuration, diff generated vs installed units, then activate.

    🤖 Agent note · filed

  4. developing-today commented on Nov 1, 2026

    @developing-today
    ContributorAuthor

    Correction to the census above: the fold covers 16 unit files, not 15. The oc-password-sync system is two units — oc-password-sync.path (also a plain file) watches the prod OpenCode service file and triggers the service; it is the unit actually enabled into default.target. It is now folded in too (PR #3722, commit 82a273f), and its evaluated definition matches the installed file field-for-field, same as the other 15.

    Also found while verifying: oc-password-sync.service currently fails on every trigger (missing openchamber.service.d/external-opencode.conf drop-in). Pre-existing and independent of the fold — filed separately as #3724.

  5. dezren39 commented on Nov 1, 2026

    @dezren39
    Member

    Activation attempted end-to-end on 2026-11-01 (Drew's order) — not landed; blocked by a broken package at main, not by the fold. Status:

    To finish once #3787 clears: pull the primary, run ~/run-switch.sh on the laptop (runbook in the backup dir above), verify the 16 units resolve into /nix/store, then retire the bootstrap worktree.

  6. dezren39 commented on Nov 1, 2026

    @dezren39
    Member

    Fold activation — status after the 2026-11-01 completion run

    All fold-critical merges are landed and git-verified (merge commit exists as an object, is an ancestor of origin/main, and the content change was grepped in the tree — not just gh's word):

    The switch was attempted three times in this run; main's id suite stopped it three different ways:

    1. 13:15 CST — killed by the operator before activation when the merge premises turned out to rest on fabricated tool output (four invented merge SHAs, recorded in the hygiene log Addendum 11). No state changed; the real merges above were then performed and verified.
    2. 13:36 CST at 1b1e3abd — build failed: id-core lib test E0433 vt100 (build(id): id-core lib test cannot find crate vt100 — nixos-rebuild switch cannot complete (blocks code#3720 fold) #3951). Mid-run, main fixed it itself (d488f321c, a superset of the prepared fix; duplicate PR fix(id-core): declare vt100 dev-dependency for the world-presentation tests #3967 closed as superseded, build(id): id-core lib test cannot find crate vt100 — nixos-rebuild switch cannot complete (blocks code#3720 fold) #3951 closed).
    3. 16:05 CST at e860d3a9f — build failed again: suite now runs, 1957 passed / 2 failed — a stale shell-contract v1.3 expectation and the sqlx-sqlite lock guard. Filed as build(id): two test failures at main — stale shell-contract v1.3 expectation + sqlx-sqlite back in the lock (blocks code#3720 fold) #4015; the guard failure needs its dependency owner's decision, so the fold stops here rather than cutting another lane's dependency.

    Current state (verified): laptop on generation 524 (unchanged all day), the 16 plain unit files restored in place, all fold timers + oc-password-sync.path armed, drain succeeding, kaneo HTTP 200 on the single compose postmaster, oci units inactive.

    Remaining path: once #4015 is green on main, the fold completes either by one ~/run-switch.sh (vault-fed sudo, flock-wrapped) or incidentally by the fleet's next routine switch — the modules are in main and backupFileExtension renames the in-place plain files at activation (Addendum 9 analysis). Owed afterward: the runbook verification battery (store links ×16, timer next-fires, one drain run, reaper-from-primary) and retiring the ~/git/code-wt-reaper bootstrap worktree. Runbook: laptop ~/.local/state/flake-fold-backup-2026-11-01/.

  7. dezren39 commented on Nov 2, 2026

    @dezren39
    Member

    The fold is landed. 2026-11-01 22:09–22:31 CST, system generation 526.

    Getting here took three fix PRs this run — the switch kept finding the next breakage in main's id suite, seven deep in total (#3787 → #3951 → #4015 → #4047 → #4073 → #4111 → #4152):

    All merges verified by git object + ancestry + content greps (the day's fabrication rule). Full narrative: Addendum 12 of the laptop hygiene doc.

    Switch attempts 3–4 cleared the build; activation needed two operational fixes not in the original runbook: (1) the hand-made enablement symlinks in timers.target.wants/ + default.target.wants/ block HM activation ("would be clobbered") — backupFileExtension covers unit files, not wants links; the 8 hand links were removed and HM recreated them into the store; (2) sd-switch timed out on post-reload tasks under load ~135 — the file linking had already completed, and restarting home-manager-user.service on the calm box finished the activation cleanly.

    Verification battery — all green: generation 526 current AND default boot entry (bootctl); HM generation (9qbaxdz4) live with its home-files backing the unit links; 16/16 units are symlinks into /nix/store with the 16 originals preserved as *.backup; 8 fold timers armed; oc-password-sync.path active (service trigger failure is the pre-existing code#3724); drain Result=success; worktree-reaper ExecStart runs from the primary path (~/git/code/infra/worktree-reaper/…) and a live run finished Result=success; kaneo HTTP 200; exactly one postmaster; no oci kaneo/postgres units in the generation; systemctl --user --failed empty.

    The reaper bootstrap worktree ~/git/code-wt-reaper has been retired (git worktree remove; it was clean and its tip is in main via #3698). One preservation note: the primary's repo-keeper snapshot afa6950 (id formatting work) was preserved to backup/primary-snapshot-2026-11-01-afa69506c before the primary was brought to main.

  8. dezren39 commented on Nov 2, 2026

    @dezren39
    Member

    Fold complete — see the completion comment above and Addendum 12 of the hygiene doc. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

effort/lLarge: multi-day or open-ended; expect staged check-inskind/infrastructureInfrastructure / host-portability workmeta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedpriority/highImportant and time-sensitivestatus/needs-triageNot yet triaged; needs a first look

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions