design(atlas): Presidio NER pass — full internal-tier + sampled secret-tier PII detection at corpus scale #6048

Parent: #6021 (classification revalidation scan). Research source: prior-art doc ~/workspace/scratch/atlas-ingestion-brief-2026-11-04/prior-art-research.md (2026-11-04). Follow-up 27 (Cluster 5).

Finding (research, in substance)

Microsoft Presidio (MIT; community-governed under data-privacy-stack) — Analyzer (regex + checksum validators + spaCy NER + context words) + Anonymizer, 40+ entity types incl. national IDs for 16+ countries, IBANs, cards. Documented caveats: person/organization names rely on NER + context and false-positive without it; some country recognizers carry high FP risk absent context words; checksum-backed recognizers (cards, IBAN) are the reliable tier. At ~107k chunks the NER pass is the expensive layer of the classification stack (the gitleaks-class regex pass is not), so its scoping is a design question, not a default.

Scope

• Full Presidio pass over the internal tier (where a leak does damage — internal is the broadly-queried tier); sampled pass over the secret tier (there PII is expected; the question is calibration, not presence).
• Runtime/cost at corpus scale on fleet hardware; batching + checkpointing shape.
• FP review queue: findings land as quarantine candidates (tier-lock upward, never silent delete — per the scanner design in the task(atlas): classification revalidation scan — verify nothing sensitive sits at the wrong tier #6021 research comment), with the matching recognizer + context attached.

Done looks like

A pass design: recognizer set, tier scoping, cost estimate, review-queue shape.