Repository navigation
design(atlas): expansive v2–v4 — feedback-driven ranking, provenance-complete resolution, self-curating corpus #6020
Description
Parent: #6018 (Atlas ingestion program). Statements S39–S40 of the 2026-11-04 brief.
Mandate (Drew, in substance)
Alongside the smaller per-topic designs, produce one larger design that expands the depth and scope of Atlas wherever the research takes it, thinking ahead to v2, v3, and v4 — not a restatement of the brief, but the brief pushed further: what does a personal corpus become when ingestion is automatic, provenance is complete, versions are kept, and every search can be judged by its user?
Starting threads (research is expected to find more)
- Feedback-trained retrieval: thumbs/essay feedback + per-stage score traces (sibling design under #6018) as training signal — per-user ranking adaptation, query-pattern evolution, and honest evaluation harnesses that distinguish "the corpus lacks it" from "the ranker buried it".
- Provenance-complete resolution: with #3125's occurrence registry + version lineage, any chunk resolves to every place it ever lived, at every version — citation-grade answers ("this claim came from file X at commit Y, lines 17–241, also present in Z").
- Cross-source lineage graph: files, embeds, sessions, issues, notes as one graph (ties to the ID graph and the zettel graph designs; writer-set authority per #5545).
- Self-curating corpus: retention driven by measured usefulness (retrieval survival + feedback), not just age — the corpus proposes its own cold-storage and deletion candidates for human approval.
- Classification-aware retrieval: tier boundaries enforced inside ranking/expansion (a secret-tier chunk never hydrates an internal-tier context), building on the classification manifest line (#5410).
- Cost-aware tiering: hot/warm/cold placement continuously rebalanced against the plan ceiling (Turso/D1/rustfs per the 2026-11-04 storage ruling).
- Agent-facing corpus: Atlas as the memory substrate for agents — session/chat/report ingestion (operations design, sibling under #6018) makes every agent's past work searchable by every future agent, with provenance spans as the trust mechanism.
Method
Survey before designing: personal-knowledge/search systems (Mem, Rewind-style capture, Khoj, Quivr, personal RAG stacks), provenance systems (W3C PROV, git-like content addressing, datalake lineage), and feedback-loop retrieval research. Label dead ends as dead ends. Each v2/v3/v4 claim must name its dependency on the v1 pieces (delta ingestion, versioning, traces) so the horizon stays buildable, not aspirational.
Done looks like
A design document in code (doc/) laying out the v2/v3/v4 arc with per-stage entry criteria, plus a follow-on issue list for anything the research shows is independently buildable sooner. Breadth is the deliverable; premature implementation commitments are explicitly out of scope.
Activity
developing-today commentedon Nov 5, 2026
CODE-1686
dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
dezren39 commentedon Nov 5, 2026
🤖 Agent note · state: handed off
dezren39 commentedon Nov 5, 2026
agent-note:v1
task: 10387
state: handed_off
lane: code-a (execution lane: opencode)
Picked up by Lane A shift (attempt 2/6). Revalidated 2026-11-05: issue #6020 is OPEN, no held / user/owned / do-not-merge / do-not-auto-edit labels. Execution lane is
opencode— handed off to the opencode claimant (repeat handoff: prior park expired unbound). Parking 1800s for the claimant window.dezren39 commentedon Nov 5, 2026
🤖 Agent note · handed off
93 remaining items
dezren39 commentedon Nov 6, 2026
🤖 Agent note · handed off