Skip to content

design(atlas): classification revalidation cadence — rule-triggered rescans, sampled human audits, allowlist expiry #6051

Open

Description

@dezren39

Parent: #6021 (classification revalidation scan). Research source: prior-art doc ~/workspace/scratch/atlas-ingestion-brief-2026-11-04/prior-art-research.md (2026-11-04). Follow-up 30 (Cluster 5).

Finding (research, in substance)

One-time scanning certifies a moment, not a corpus — that is a named dead end. Rules improve, corpora get re-tiered, and sources turn out dirtier than believed, so revalidation is a recurring gate: new rules find old leaks. The cadence design has three moving parts:

  • Rule-update-triggered rescans: when the scanner rules / inventory pack / recognizer set changes, the affected scan re-runs over the corpus (deterministic layer in full; NER layer per its scoping design, sibling under task(atlas): classification revalidation scan — verify nothing sensitive sits at the wrong tier #6021).
  • Sampled human audit per round: a small random sample of each tier hand-checked, to estimate the scanners' miss rate instead of assuming it. "No detector catches everything" is every tool's own disclaimer; the audit sample turns that caveat into a number.
  • Allowlist expiry: allowlist entries carry reasons and expiry; a stale allowlist is how leaks get grandfathered.

Scope

This design takes #6021's recurrence recommendation (one-off vs standing gate) as its input and specifies the standing form: triggers, per-layer scope per trigger, audit sample sizes, and where the cadence plugs into the ingestion-operations design (#6019) as a standing gate.

Done looks like

A cadence spec: trigger table, scope per trigger, audit method with miss-rate estimate, allowlist lifecycle.

Activity

  1. added
    area/infraInfrastructure, deployment, and operations
    kind/designA design proposal (design-architect): multi-step, phased, decision-bearing
    on Nov 5, 2026
  2. developing-today commented on Nov 5, 2026

    @developing-today
    Contributor
  3. added
    queue/queuedHas a queued task in the Supabase agent queue
    task/triageQueue packet kind: verify, reproduce, and route this item
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 5, 2026
  4. dezren39 commented on Nov 5, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · done

    Triage findings (verified fresh 2026-11-05)

  5. added
    status/triagedTriaged and routed (see agent note)
    and removed
    queue/claimedQueue worker lifecycle state
    work/assignedQueue worker lifecycle state
    on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/infraInfrastructure, deployment, and operationskind/designA design proposal (design-architect): multi-step, phased, decision-bearingstatus/triagedTriaged and routed (see agent note)status/waiting-dependencyWaiting on a dependencytask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions