Skip to content

Align Stage 2 filtering and precision claims with current behavior #675

Description

@Spectorian

Problem

The README describes Stage 2 as filtering false positives and improving precision to approximately 87%. The linked implementation retains deterministic findings when the model disputes or does not confirm them. Meta-review can annotate and enrich findings, but those findings remain in the result.

An accuracy figure without a clear benchmark, model/scanner version and evaluation boundary can also be mistaken for the current pipeline's validated performance.

Expected documentation

Describe the current review/enrichment behavior and its limits where Stage 2 is explained. Either connect the number to a verified, clearly scoped evaluation or remove the unqualified number. Do not substitute a rate calculated from selected false-positive examples.

The documentation should describe that behavior accurately, without changing how LLM disagreement affects deterministic findings.

Historical #540 corrected a different research-denominator claim; it does not resolve this Stage 2 wording.

Relevant code

README.md:815, meta_analyzer.py:381, meta_analyzer.py:437.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions