Title & Overview

Template: Named Entity Recognition (NER): An Intermediate, End-to-End Analysis Tutorial
Overview (≤2 sentences): Learners will implement NER pipelines using both rule-based/spaCy approaches and transformer-based token classification models. It is intermediate because it requires handling token–label alignment, reproducibility, and error slicing across entity types.

Purpose

To teach learners how to build, evaluate, and compare defensible NER baselines versus transformer models, and how to interpret performance differences by entity type, span length, and domain. This emphasizes error analysis and reproducibility, not just getting entities out of text.

Prerequisites

• Skills: Python, Git, pandas, ML basics.
• NLP: tokenization, embeddings, BIO tagging, evaluation metrics (F1, precision, recall).
• Tooling: pandas, scikit-learn, spaCy, Hugging Face Transformers, seqeval/torchmetrics, MLflow, FastAPI.

Setup Instructions

• Environment: Conda/Poetry (Python 3.11), deterministic seeds.

• Install: pandas, scikit-learn, spaCy, Hugging Face Transformers + Datasets, seqeval, MLflow, FastAPI.

• Datasets:

  • Small: CoNLL-2003 (English NER: PER/LOC/ORG/MISC).
  • Medium: OntoNotes 5 (broad domains, multiple entity types).

• Repo layout:

  tutorials/t4-ner/
    ├─ notebooks/
    ├─ src/
    │   ├─ data_utils.py
    │   ├─ models.py
    │   ├─ train.py
    │   └─ config.yaml
    ├─ data/README.md
    ├─ reports/
    └─ tests/
  

Core Concepts

• BIO tagging scheme: Begin/Inside/Outside labeling of entity spans.
• Byte-level BPE tokenization: alignment challenges when subwords split entities.
• Classical vs neural approaches: spaCy’s rule-based + statistical vs transformer token-classification (e.g., DistilBERT).
• Evaluation granularity: entity-level F1 vs token-level accuracy.
• Error slicing: entity-type F1, short vs long entities, capitalization sensitivity.

Step-by-Step Walkthrough

1. Data intake & splits: load CoNLL-2003/Ontonotes; stratified reproducible splits.

2. Classical baseline: spaCy NER pipeline (out-of-the-box, fine-tune optional).

3. Transformer baseline:

   • DistilBERT (byte-level BPE).
   • Token classification head (linear layer on top).
   • Training with label alignment (map subword pieces to labels).

4. Evaluation: entity-level precision/recall/F1 (seqeval); token-level accuracy as sanity.

5. Error analysis: per-entity-type F1, confusion examples (ORG vs LOC), span length errors.

6. Reporting: metrics tables, error slices, example entities to reports/t4-ner.md.

7. (Optional) Serve: FastAPI endpoint that returns entity spans and types, validates input, and logs requests.

Hands-On Exercises

• Ablations: spaCy vs transformer; different max seq-len (128 vs 256); freeze vs full fine-tune.
• Robustness: test casing (UPPERCASE, lowercase), punctuation noise, domain-shift text.
• Slice analysis: compare performance across PER, LOC, ORG, MISC classes.
• Stretch: extend to multilingual NER with XLM-R.

Common Pitfalls & Troubleshooting

• Token–label mismatch: subword splits → alignment errors if not handled.
• Metrics misuse: token-level accuracy inflates performance vs entity-level F1.
• Over-cleaning: lowercasing can erase entity cues.
• OOM: long docs; mitigate with truncation or sliding windows.
• Silent drift: tokenizer vocab changes alter entity alignment — must log artifacts.

Best Practices

• Track entity-type performance, not just overall.
• Log config, seeds, tokenizer vocab/merges, dataset fingerprints with MLflow.
• Unit tests: token-label alignment correctness, deterministic splits.
• Serve with schema guardrails: max length, allowed entity types, input validation.
• Keep Baseline → Transformer narrative clear.

Reflection & Discussion Prompts

• Why do transformers often outperform spaCy NER on rare/multi-token entities?
• How does normalization (lowercasing, NFKC) affect entity recognition?
• What’s the trade-off between entity recall and precision in real-world applications?

Next Steps / Advanced Extensions

• Apply domain adaptation (fine-tune on civic-text corpus).
• Experiment with nested entities and overlapping spans.
• Explore active learning: identify hardest sentences for annotation.
• Lightweight deployment: batch inference + drift monitoring.

Glossary / Key Terms

BIO tagging, entity-level F1, token–label alignment, byte-level BPE, DistilBERT, seqeval, slice analysis.

Additional Resources

• [spaCy](https://spacy.io/)
• [Hugging Face Transformers](https://huggingface.co/docs/transformers)
• [Hugging Face Datasets](https://huggingface.co/datasets)
• [seqeval](https://github.com/chakki-works/seqeval)
• [MLflow](https://mlflow.org/)
• [FastAPI](https://fastapi.tiangolo.com/)

Contributors

Author(s): TBD
Reviewer(s): TBD
Maintainer(s): TBD
Date updated: 2025-09-20
Dataset licenses: CoNLL-2003 (LDC license), OntoNotes (LDC license).

Issues Referenced

Epic: HfLA Text Analysis Tutorials (T0–T14).
This sub-issue: T4: Named Entity Recognition (NER).