You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In SCANNER_AUGMENTATION_PLAN.md, item P0 highlights a critical precision dynamic in the scanner pipeline:
Currently, in openhack/agents/validator_swarm.py:32, model = settings.validator_model_id or self.llm.model. When validator_model_id is unset, the validation swarm defaults to using the identical model as the hunter swarm ("self-validation" / model grading its own work). In benchmark research (Benchmarks/lean-harness), this self-validation causes precision inflation and confirmation bias.
Additionally:
Tying votes (e.g. 1 confirmed vs 1 rejected) should not default to confirmed.
Common false-positive patterns in static findings (CSRF where cookies are SameSite=Lax/Strict, XXE where parsers do not expand external entities like xml2js/sax, XSS in auto-escaped JSX text nodes, and framework development default configurations) should have mandatory hardened validation checks.
Proposed Implementation
Cross-Model Validation Resolution:
Resolve validator models to independent models by default (e.g., hunter gpt-5.6-sol / glm-5.2 -> validator kimi-k2.5).
Add a startup check and session trace event warning if the validator model is identical to the hunter model.
Hardened Exploitability Prompts in VALIDATOR_PROMPT:
CSRF: Require inspection of cookie configuration and middleware; confirm only if auth cookie is NOT SameSite=Lax/Strict.
XXE: Confirm only if parser resolves external entities.
XSS: Confirm only if untrusted input reaches a raw-HTML sink (dangerouslySetInnerHTML, v-html, etc.), auto-escaped JSX is false_positive.
Framework dev configs (Nuxt DevTools, Vite HMR) and blank rationale -> false_positive.
Clarify false_positive strictly means mitigated or unexploitable (low-but-real remains confirmed low).
Strict Majority Consensus Voting:
Configurable validator votes (validator_votes, e.g. 3 for 2-of-3 majority).
Strict majority required (>= (votes // 2) + 1). Ties are strictly rejected as false positives.
I have an implementation ready with unit tests in feat/cross-model-validation-p0 and will link the PR to this issue once opened.
Activity