LLMs Exhibit Confirmation Bias In Vulnerability Detection

Researchers in an arXiv preprint (Mar 19, 2026) find that LLM-based code review exhibits confirmation bias, through two studies: a controlled experiment on 250 CVE/patch pairs across four models and adversarial pull-request tests. Framing changes as bug-free reduced vulnerability detection rates by 16–93%, and adversarial PRs bypassed detection 35% against GitHub Copilot and 88% against Claude Code. Metadata redaction and explicit instructions largely restored detection.