Repository navigation
Atlas step 5: nix packaging (pkgs/atlas-search) + amd deploy #5355
Description
Step 5 of the Atlas binary+service epic (#5350). Design of record: doc/2026-11-04T08-07-13Z_design_atlas_binary_service/ (§4 packaging/deploy, §7 Q1).
Scope
pkgs/atlas-search/default.nix:rustPlatform.buildRustPackagewithcargoLock.lockFile, pkgs/id pattern. Two hard requirements carried from the fork's native-embeddings work (opencode#1554/id(web): mirror tight attr into richSchema bullet/ordered lists (#1469) #1559) — do not rediscover them:- ONNX Runtime is dynamically loaded from the nix closure (the
ortcrate must not download a prebuilt runtime at build or run time). - The closure must carry
libstdc++.so.6andlibz.so.1— verified by closure inspection + a smoke run, not assumed.
- ONNX Runtime is dynamically loaded from the nix closure (the
- Post-install smoke check in the derivation: start the binary, run one FTS-only and one embedded query against a fixture DB.
- Model provisioning:
atlas-search fetch-modeldownloads the pinned HuggingFace revision ofsnowflake-arctic-embed-m-v2.0into~/.local/state/atlas-search/model/<rev>/, verifying sha256 hashes recorded as constants in the repo; atomic rename into place; rerun is a no-op when the pinned set verifies. (Design §7 Q1 recommends state-dir over a nix-store FOD; if Drew flips Q1 before this step starts, implement the FOD instead and note it here — the rest of the design is unaffected.) - Deploy on amd: systemd user service declared in home-manager (
atlas-search.service,ExecStart=serve,Restart=on-failure), landed disabled.tailscale serveexposure on the MagicDNS name in front of 127.0.0.1. Proposed port 18760 — verify against the fleet port inventory (check-proxy-port-fallbacks.sh,home/common/default.nix) and take the next free 187xx if taken; record the final port on this issue. - State dir
~/.local/state/atlas-search/holdsservice.json(0600),model/, logs. The deploy provisions structure, not real client tokens (token generation is step 6, with Drew).
Dependencies: step 3 (CLI), step 4 (serve).
Verification gate (exit criteria)
nix buildof the package green on the laptop (never the sandbox); the smoke check runs inside the build.- A laptop switch is not required to close this step: the package + disabled unit land on main; activation is step 6. If a switch is ridden anyway, the unit must come up disabled and the Python must remain the resolved
atlas-search.
Out of scope: enabling the service, issuing tokens, any default flip (all step 6).
Acceptance criteria
pkgs/atlas-searchbuilds green viarustPlatform.buildRustPackageon the laptop (never the sandbox), with the in-derivation smoke check (FTS-only + embedded query against a fixture DB) passing.- ONNX Runtime is dynamically loaded from the nix closure (no prebuilt download at build or run time); closure inspection verifies
libstdc++.so.6andlibz.so.1are present, plus a smoke run proves it. atlas-search fetch-modelprovisions the pinned HuggingFace revision ofsnowflake-arctic-embed-m-v2.0into~/.local/state/atlas-search/model/<rev>/with sha256 verification and atomic rename; a rerun is a no-op.- The home-manager
atlas-search.serviceuser unit lands on main DISABLED; tailscale serve exposure is declared in front of 127.0.0.1 on the verified-free 187xx port (proposed 18760), and the final port is recorded on this issue. - No real client tokens are provisioned (step 6); the Python remains the resolved
atlas-searchafter this step.
Activity
atlas-search step 1: atlas-embed crate + Gate A fixtures (#5351)
dezren39 commentedon Nov 4, 2026
Intake review — fleet monitor, 2026-11-04.
Related: epic #5350; depends on step 3 (CLI) and step 4 (#5354, serve mode). The two closure hard requirements come from the fork's native-embeddings work (opencode#1554/#1559) — already paid for once; the acceptance criteria carry them so they are not rediscovered.
Notes: design §7 Q1 (state-dir model vs nix-store FOD) is Drew's to flip before this step starts; the issue records the recommendation (state-dir). Port 18760 must be verified against the fleet port inventory, not assumed.
Risks: low-to-moderate — packaging work with a disabled landing; activation risk is zero by construction (unit lands disabled, Python stays resolved).
Suggested approach: package + smoke check first, closure inspection second, then the disabled unit and serve declaration; record the final port here.
atlas-search: fetch-model provisioning verb + baked ONNX Runtime defa…
pkgs/atlas-search: nix package + standalone flake (step 5, #5355)
home: atlas-search serve unit, landed disabled (step 5, #5355)
Merge pull request #5439 from developing-today/atlas-step5-packaging
dezren39 commentedon Nov 4, 2026
Step 5 complete — landed DISABLED via PR #5439, merge commit
2b39005876b503ea69010640356b2df35fbe1b2d.Delivered
atlas-search fetch-model(Q1 ruling implemented): pinned HF repo/rev (Snowflake/snowflake-arctic-embed-m-v2.0@95c2741480856aa9666782eb4afe11959938017f), in-source sha256 for both files, streamed+hashed download into a staging dir, atomic rename promote into~/.local/state/atlas-search/model/<rev>/, verified no-op on re-run, loud refusal on mismatch. A mkdir provisioning lock was added during verification after orphaned concurrent runs were caught racing on the shared staging dir (they could interleave writes into the same files); a killed run leaves the lock + staging behind and the next run's error says how to clear them — demonstrated live when the laptop rebooted mid-download (fleet switch): state recovered cleanly, no corrupt snapshot was ever promoted.pkgs/atlas-search(+ standalone sub-flake):buildRustPackageon the pinned rust-overlay toolchain; closure carries onnxruntime-1.27.1, gcc lib (libstdc++.so.6), zlib (libz.so.1) — verified withnix-store -qR; the ORT path is baked in viaATLAS_SEARCH_ORT_LIB_DEFAULT.doInstallChecksmoke: fixture FTS + hybrid envelopes byte-equal to the Python recordings, ORT-load probe passes (init_from succeeds from the closure before the tokenizer error).systemd.user.services.atlas-searchin home config: Service+Unit only, no Install section — cannot be enabled until step 6. Port 18760 inventoried free (repo range assigns 18742–18751; live listeners checked). Tailscale serve mapping is recorded as a step-6 activation action in the unit comment (serve state is imperative tailscaled state; no declarative precedent in the repo — only t3's--tailscale-serveflag exists).Verification (packaged artifact, clean env)
/nix/store/kvl2w87san8zily8hhr06zkb77gbrsq7-atlas-search-0.1.0(rebased onto current main incl. fix(atlas-search): escape U+007F in pyjson ensure_ascii mode (#5404) #5405/fix(atlas-search): validate CLI counts instead of clamping to 0 (#5407) #5411; workspace tests green, clippy/fmt clean); home eval's ExecStart converges to this exact output.env -i(HOME+PATH only) standalone search vs production read-only, back-to-back with the Python CLI: top-10 exact match ("how do we back up the database"). A 3-question pass agreed on 2 immediately; the third differed on first pass and matched exactly on a controlled back-to-back re-run — the corpus is being rewritten live by the index work (chunk ids moving), so the first-pass delta was corpus churn, not a packaging defect.systemctl --user is-enabled atlas-search→ not-found on the live system, no~/.config/atlas-search/on the real home, Python wrapper at~/.local/bin/atlas-searchuntouched and still the resolved CLI.Environment note: the laptop's GitHub credential died with the reboot (keyring locked, no desktop login), so the branch was pushed via the sandbox's authenticated gh after fetching the branch over the SSH tunnel; merge verified by read-back.
Step-6 (#5356) readiness
Install.WantedBy+ switch, runinitfor real on the laptop (0600 service.json already designed; token VALUES never leave the laptop except the client.json distribution Drew approves for sandbox/OpenChamber), create the tailscale serve mapping for 18760.personalget tokens in v1: drew-cli is the secret-bearing one).dezren39 commentedon Nov 4, 2026
🤖 Agent note · done