Repository navigation
design(atlas): schema-driven filter surface from the classification manifest (post-cutover) #5410
Description
Follow-on to epic #5350 (Atlas binary + service), to be scheduled AFTER step 6 (cutover). Design only — no implementation in this issue.
Where things stand
atlas-core's filter surface is split: the vocabulary is data-driven (filters match the labels the classification pass wrote onto chunks — kinds, topics, projects, clients — so new labels are filterable immediately), but the dimensions and semantics are hand-coded in four places (atlas-core filters module + SQL builders, the CLI flag surface, atlas-proto request types, serve-mode validation). Adding a new classification dimension today (e.g. a source-app label) requires a code change in each place. Match semantics are likewise code: exact match today, with LIKE-pattern matching tracked separately in #5375.
Proposal (for a design note)
A classification manifest — carried with the corpus publish metadata — declares the filterable dimensions and their match semantics (exact / pattern / range), and atlas-core builds its filter surface from that manifest, with the CLI and service inheriting the same surface through atlas-proto. A reclassification pass could then change what searches can slice by without a binary release.
Hard constraint
The sensitivity ladder and ceiling semantics stay CODE-defined, never data-defined. The ceiling is a safety property enforced inside branch SQL (filter-before-rank); a mislabeled or corrupted row must never be able to redefine its own visibility. Only dimension declarations and match semantics for non-sensitivity fields are manifest material.
Open questions for the design
- Manifest home: publish-file metadata vs a DB table vs both (local mode vs Turso mode must agree).
- Versioning + failure mode: manifest unreadable/mismatched → fall back to the built-in surface, loudly, never silently widen.
- Interaction with atlas-core: topics/projects filters are unescaped SQL LIKE patterns — % and _ act as wildcards, a quote crosses JSON element boundaries #5375's pattern filters (land exact+pattern semantics once, in the manifest, rather than twice).
- Dedupe-key impact: serve-mode Computations key on the effective filter tuple — a manifest change must version into the key.
Acceptance for this issue: a design note under doc/ following the epic's design-note pattern, with a go/no-go recommendation.
Activity
dezren39 commentedon Nov 5, 2026
Triage outcome — task 10035 (2026-11-05, lane code-p)
Verdict: CONFIRMED and unblocked — design path scoped; follow-up design-note task enqueued.
Precondition met. Issue #5410 is design-only, scheduled AFTER step 6 (cutover) of epic #5350. Fresh check: step issues #5351–#5356 are all CLOSED, and the epic's 2026-11-04T17:20Z status comment records the cutover landed (Gate B passed, Δhits@10 = 0, ΔMRR = +0.0016; service live on the laptop). The epic itself stays open only for the deferred index-backend default decision, which does not gate this design.
Still true on current main (
04f78696ab, 2026-11-05): the filter surface is hand-coded exactly as this issue describes —pkgs/atlas-search/crates/atlas-core/src/filters.rsdefinesFilters { kinds, topics, projects, clients, max_sensitivity, since, until }in code, with the sensitivity ladder (SENS_ORDER) code-defined as the hard constraint requires. A grep for a classification/filter manifest acrosspkgs/atlas-search/finds nothing, anddoc/holds only the binary/service design (2026-11-04T08-07-13Z_design_atlas_binary_service) and the scaler revision — no manifest design note exists yet. Nothing supersedes this issue; the only duplicate search hits are #5410 itself.Interaction to settle in the design, not before it: #5375 (unescaped LIKE patterns in topics/projects filters) is still OPEN. Per this issue's own open questions, exact+pattern semantics should be landed once, in the manifest — so the design note must either incorporate #5375's semantics decision or explicitly sequence against it; it is an input, not a blocker.
Routing: follow-up task enqueued for the design note itself (acceptance = this issue's own: a design note under
doc/following the epic's design-note pattern, with a go/no-go recommendation, covering manifest home/versioning/failure mode, the #5375 interaction, dedupe-key versioning, and the invariant that sensitivity/ceiling semantics stay code-defined). No implementation is authorized by that task — the note's go/no-go gates any build work.agent-note:v1
{"kind": "triage", "task": 10035, "issue": 5410, "state": "done", "outcome": "confirmed_unblocked_design_path_scoped", "evidence": "steps #5351-#5356 CLOSED; filters.rs on main 04f7869 hand-codes Filters dimensions; no manifest in pkgs/atlas-search; no design note in doc/"}
doc(atlas): address Codex review — decoded-element equality for exact…
Merge pull request #6223 from developing-today/queue/10475-5410-manif…
dezren39 commentedon Nov 5, 2026
🤖 Agent note · done
doc/following the epic's design-note pattern, with a go/no-go recommendation).doc/2026-11-05T13-03-49Z_design_atlas_manifest_filter_surface/. Recommendation: GO, with the §8 phasing. Covers manifest home (oneclassification_manifesttable, both modes, same transaction as labels), versioning + loud built-in fallback, the atlas-core: topics/projects filters are unescaped SQL LIKE patterns — % and _ act as wildcards, a quote crosses JSON element boundaries #5375 interaction (its escaping fix lands FIRST as the standalone implementation of the note's §5 exact semantics), dedupe-key versioning (CacheKeygains(manifest_version, content_hash)), and the hard invariant (sensitivity/ceiling stay code-defined; no sensitivity keys in the schema).json_listmatching now specified as decoded element equality (JSON-encode the value before LIKE-escaping); (2) ageneric_listgeneric label store added so a new dimension is data end-to-end (the phase-5 manifest-only gate is now attainable). Round-2 review: no new findings.