Skip to content

migrate all Atlas/search/embedder scripts into developing-today/code + fleet binary; retire outside copies #5670

Open

Description

@dezren39

Implementation of §8 of the Scaler-era data architecture revision (design doc PR #5667, doc/2026-11-04T21-27-34Z_design_atlas_scaler_revision/). Drew's order: everything Atlas/search/embedder migrates into this repo and the fleet binary; nothing canonical lives outside.

Live items → destinations (per §8 table):

  • hatch-personal/workspace/turso-atlas/scripts/atlas_search.py → pkgs/atlas-search/ as the permanently-kept Python eval/reference implementation.
  • eval3.py + golden-311 corpus → eval tooling beside pkgs/atlas-search.
  • embed_pass3.py, embed_onnx.py, atlas-ingest.py → absorbed by the aggregator's embedding/promotion stages (tracking issue for the staging tier).
  • ~/tools/bin/atlas-search wrapper → retired once the fleet search domain entry + serve role are the production path.
  • atlas-backup (laptop systemd unit; nix-declared) → fleet search backup verb / fleet serve maintenance role; first verify the flake copy matches the surviving ~/tmp/atlas-backup.sh shell original.
  • Snowflake model provisioning artifacts → the Atlas step 5: nix packaging (pkgs/atlas-search) + amd deploy #5355 nix provisioning path.

Historical → retire/archive after migration verifies:

  • Laptop ~/atlas/v3/swap-2026-11-03/ one-off swap/gate/remediation scripts + staging.db — archive as the hatch-personal#20 evidence record after the in-flight index rewrite completes and verifies (do not touch while the rewrite runs).
  • Sandbox transfer artifacts (atlas-model-rev.tar 1.2 GB, atlas-step6-closure.nar 300 MB, ~/tmp/atlas-migrate/, ~/tmp/atlas-parts/, proof logs) — delete once model + closure are confirmed reproducible from this repo (~1.5 GB reclaimed).
  • hatch-personal/workspace/turso-atlas/docs/ design history — archive in place (hatch-personal is the record).

Acceptance: every §8 row has a landed destination or a recorded archive/delete commit; grep over hatch-personal + ~/tools + ~/atlas finds no remaining live Atlas script outside this repo; the eval harness runs from its new home with golden-311 parity.

Design: #5667 §8. Parent context: hatch-personal#20/#21.

Activity

  1. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  2. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    work/assignedQueue worker lifecycle state
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    queue/claimedQueue worker lifecycle state
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/queuedHas a queued task in the Supabase agent queue
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    on Nov 5, 2026
  3. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  4. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  5. 137 remaining items

  6. added
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    queue/claimedQueue worker lifecycle state
    on Nov 6, 2026
  7. dezren39 commented on Nov 6, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  8. dezren39 commented on Nov 6, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  9. dezren39 commented on Nov 6, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · handed off

  10. added
    queue/claimedQueue worker lifecycle state
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    status/parkedDeliberately paused with a recorded resume condition
    and removed
    status/parkedDeliberately paused with a recorded resume condition
    queue/parkedQueue task is parked (waiting on CI, time, or an event)
    queue/claimedQueue worker lifecycle state
    on Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

queue/parkedQueue task is parked (waiting on CI, time, or an event)status/parkedDeliberately paused with a recorded resume conditiontask/small-fixQueue packet kind: bounded fix with reproduction and targeted testswork/assignedQueue worker lifecycle state

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions