Artificial intelligence can search enormous codebases faster than any human security team. That does not mean every warning it produces deserves a developer’s attention. In vulnerability management, an unverified finding is not free: someone still has to reproduce it, judge its severity and decide whether it is real.
Google’s newly detailed PageBreak project points toward a better standard. The internal agent looks for web vulnerabilities, but its most important design choice is what happens after discovery. Google says unverified candidates are kept away from product teams while deterministic tools attempt to prove that an issue is exploitable.
A plausible explanation is not a security result
Large language models are excellent at producing convincing narratives. In security work, that strength can become a liability. A report may describe a believable attack path while quietly assuming access, data flow or runtime behaviour that does not exist. If hundreds of those reports reach engineers, automation has not removed toil; it has redistributed it.
The useful unit of output is therefore not an alert. It is evidence: a reproducible request, an observed response, a minimal exploit path and enough context for an owner to verify the result. Models can still form hypotheses, but a validation layer should separate ideas worth investigating from claims ready to interrupt a product team.
Confidence needs an honest label
Not every flaw can be proven automatically. Some require unusual state, specialized access or a complex chain across services. The answer is not to hide uncertain findings or present them as facts. Systems should label them as leads, explain which capability is missing and feed that information back into future tests.
That distinction also creates better metrics. Counting generated reports rewards volume. Tracking verified vulnerabilities, false-positive rates, time to reproduce and time to remediation rewards outcomes. Security leaders buying AI tools should ask for those measurements before they ask how many issues a scanner can find in an hour.
Prevention still beats an automated chase
PageBreak also reinforces an older lesson: secure-by-design frameworks can remove entire bug classes before a scanner enters the picture. Google reports that applications built on its high-assurance web frameworks produced only two cross-site scripting findings across hundreds of applications as of September 4, 2026, both limited to internal or debug endpoints with hardening gaps.
AI can expand the reach of security testing, but it should not turn engineering teams into fact-checkers for machine speculation. The winning systems will be the ones that arrive with receipts—and know when they do not have them.
