GitHub improves secret scanning verification with LLM reasoning

In a June 11, 2026 post on the GitHub Blog, Mariko Wakabayashi - a Principal Applied Scientist at Microsoft who leads agentic AI workflows for cybersecurity operations - described a collaboration between GitHub and Microsoft Security & AI's Agents Offense team to add context-aware LLM reasoning to the verification step of GitHub's secret scanning pipeline. The work adapts the verification approach from Agentic Secret Finder, a detection and verification system built to judge potential secrets in context rather than by pattern match alone. GitHub's existing pipeline combines pattern-based detection (known provider token formats) with AI-powered generic detection for unstructured secrets like passwords; this change inserts an LLM reasoning stage between candidate detection and alerting, evaluated against customer-confirmed false positives with a target of a 65% reduction. The measured result was a 75.76% reduction, which GitHub says was achieved without changing upstream detection logic or reducing coverage.

GitHub reports the key design choice was giving the model better context rather than more context: instead of passing entire files or repositories to the LLM (which GitHub says adds noise, cost, and latency), the system extracts a small set of usage signals - for example, whether a candidate value is assigned to a variable and later passed into an API request, an authentication header, a database client, or a cloud SDK call. That usage pattern helps the model distinguish an actual credential from a placeholder, test fixture, random UUID, or other opaque string that merely looks secret-shaped. In most cases GitHub says this file-level context is enough, without needing repository-wide analysis. GitHub already operates secret scanning at large scale, processing billions of pushes and covering tens of millions of developers across millions of repositories, per the company's own account of its baseline system.

The reported trade-off - focused, file-level usage signals over broad codebase context - is a directly transferable design pattern for teams building LLM-based verification or triage layers on top of any existing rule-based or ML detector, whether in security, fraud, or content-moderation pipelines: the verification model does not need to see everything the base detector saw, it needs the specific signals that resolve ambiguity. GitHub frames this as an add-on filtering stage rather than a detector replacement, which keeps the blast radius of a model change contained to alert quality rather than detection coverage - a useful pattern for teams wary of introducing LLM components into detection paths that need to preserve recall guarantees.

GitHub says it is continuing to evaluate the approach on larger datasets and live production traffic, so watch for a follow-up post with real-world (as opposed to evaluation-set) precision and recall numbers. The original post does not specify model choice, hosting (on-premise, hosted API, or hybrid), or per-alert latency and cost, which are the details security engineering teams would need to reproduce or budget for a similar verification layer.

This is currently a single-source, vendor-reported result: GitHub evaluated its own system against its own target and has not yet published independent or live-traffic numbers, so the 75.76% figure should be read as GitHub's internal evaluation-set result rather than an externally audited benchmark. Reducing alert fatigue with a context-aware LLM verification layer bolted onto an existing detector is a pattern likely to recur across security and fraud-detection tooling generally, since it lets teams improve precision without re-architecting or retraining upstream detection systems.