AI Security Scanning Shifts the Bottleneck to Verification and Patching

Eugene Yan says security teams should prepare for a shift in where software-security work accumulates. Frontier models can identify vulnerabilities at a volume that makes the downstream work—proving, prioritizing, routing, and fixing them—the practical constraint.

The capability trend is visible in cybersecurity time-horizon evaluations, Yan says. The UK AI Security Institute’s chart showed models completing increasingly long cyber tasks, including reverse engineering and web exploitation, with a step up from the prior trend line. On-screen, the institute estimated that model capability on cyber tasks was doubling roughly every five months, while marking its scaling estimate as low confidence.

Mozilla’s Firefox data offered a more operational example. Monthly security fixes were around 20 for much of 2025, rose to 61 and 70 in February and March 2026, then reached 422 in April. Yan described that as roughly 20 times the prior year’s monthly average and said Mozilla attributed about two-thirds of the April fixes—about 271—to METR’s preview model.

Failures in widely used dependencies still demonstrate why this matters: Yan briefly pointed to Log4Shell and Heartbleed as examples of flaws whose effects extended far beyond the codebases where they originated. His point was not that models remove that risk, but that they can now surface defects at a scale that changes the work required after discovery.

Anthropic’s own open-source scanning effort made the imbalance visible. Yan said the work scanned more than 1,000 repositories and produced 23,019 candidate findings. The displayed results are not a conventional linear funnel: they show distinct external-review and direct-to-maintainer reporting paths, alongside aggregate disclosure and remediation counts. But the overall pattern was clear—candidate generation vastly exceeded the capacity to validate, communicate, and remediate findings.

The important operational question is therefore not whether a model can produce more reports. It is whether an organization has a system for turning a large, uneven stream of candidates into trustworthy, owned, and deployed fixes.

Yan’s proposed operating model has six steps. Two are upfront investments per codebase: establish a threat model and build a sandbox. Four recur as a cycle: discover potential issues, verify whether they are exploitable, triage the confirmed set, and patch the underlying problems.

Mozilla’s experience, as Yan presented it, captures why the surrounding harness matters. Early model-based experiments showed promise but produced too many false positives to scale. Agentic harnesses changed the result because they can detect genuine security issues and discard speculation that cannot be reproduced. The model is not just asked to inspect code; it is given a defined environment, tools, constraints, and a process for testing its conclusions.

The six steps also impose different optimization goals. Discovery is designed for recall: it should cast a wide enough net not to miss real vulnerabilities. Verification is designed for precision: it should distinguish demonstrated exploits from false alarms. Triage ranks the remaining work for people who must act on it. Patching supplies feedback for the next scan.

The setup is not a preliminary formality. The threat model supplies context that later determines whether a technical defect is meaningful, while the sandbox gives the system a controlled place to demonstrate whether it is exploitable. Both support every pass through the cycle, whether a team runs the process periodically, after a new feature, or against a new commit.

For Yan, the threat model is the highest-leverage early step because source code alone does not explain a system’s real security posture. One team he cited found that, where threat models and design documents were well documented, model findings were exploitable 90 percent of the time. Yan called anything above 75 percent a strong outcome.

A CISO’s formulation summarized the problem: the model has strong context about the code but weak context about “us.” It can read a route handler, database call, commit history, and previous patch. It cannot infer from code alone why a service was designed a particular way, what on-call engineers have repeatedly observed, who can reach it, or which controls sit outside the application.

A threat model should turn that implicit knowledge into usable context. Yan described bootstrapping one from source code, internal documents, Git history, previous patches, and past CVEs. A model can use those materials to infer where related weaknesses may still exist. But it should also interview a system owner: Which inputs are trusted, and why? Who can reach the service? What would compromise of this component mean? Which risks are already mitigated elsewhere?

In Yan’s running example, an order-lookup service exposes customer data including email addresses, shipping addresses, line items, and totals. Its entry point is a request such as GET /orders?id=<order_id>. The threat model labels the query parameter untrusted, the Postgres connection trusted, and names SQL injection and missing access control as threats to evaluate.

The sandbox exists for isolation and reproducibility. Agents may inspect untrusted code or deliberately develop proof-of-concept exploits, so Yan’s baseline is a VM or similarly isolated environment with egress locked down and without cloud credentials, tokens, or production secrets. Constraints should be enforced in configuration rather than entrusted to a prompt.

Reproducibility matters because a finding can depend on an old dependency, a particular commit, or an environment configuration that later agents do not share. Pinning image tags, commit SHAs, and dependency versions, then snapshotting the environment, gives each scan and verification attempt a common baseline.

Yan acknowledged that representative sandboxes are difficult to build. But he described them as the “biggest efficacy lever” reported by one offensive-security team: models were given test beds with live systems where they could run and detonate proof-of-concept exploits. The team’s practical rule was simple: a candidate counted as a true positive only if the agent could build and run a PoC against the test bed.

For the order service, the sandbox used three Docker images: the Python/Flask application, Postgres seeded with fake PII, and Redis. A security agent sat outside the target boundary and probed the application over HTTP. The target was configured without network egress.

This setup lets a team demonstrate impact without using production as an experiment. It also makes independent verification possible: a verifier can begin in a clean environment rather than inheriting the discovery agent’s state or reasoning.

Discovery works best when models receive the kind of material a new senior security engineer would get on the first day: the threat model, architecture documentation, past CVEs, code-search capability, request and response tools, traffic logs, and the ability to run scripts. Yan’s argument is that a model should not be expected to read source code in isolation and guess whether a route is reachable or an exploit succeeds. It should be able to test candidates as it investigates them.

Prompting should become less prescriptive as models improve, he said. Yan found himself cutting prompt length substantially with each capability jump. Rather than naming every vulnerability category to inspect, a more capable model can be given a goal—for example, finding where untrusted data crosses a trust boundary—and infer the investigation.

The order-service example is deliberately small:

@app.get("/orders")
def get_order():
    order_id = request.args["id"]
    row = db.execute(f"SELECT * FROM orders WHERE id = '{order_id}'")
    return jsonify(row)

The discovery agent identifies a CWE-89 SQL injection: unsanitized user input flows into an f-string SQL query. A crafted identifier could append a UNION SELECT query and retrieve data from tables available to the database role. Yan said current models are already good at finding straightforward flaws like this, including in code a non-security engineer might reasonably write.

Verification has the opposite job. Yan recommends an independent, adversarial verifier that sees the finding but not the discovery agent’s reasoning traces. It should begin with the assumption that the vulnerability is false, run in a fresh container, and try to prove or disprove exploitability. Having the discovery agent also validate its own work can make it self-censor, reducing recall.

In the order-service sandbox, the verifier receives the suspected vulnerable line, constructs a curl request containing a SQL-injection payload, and retrieves email addresses, phone numbers, and addresses from the users table. The PoC firing converts a plausible code-level flaw into a demonstrated data-exfiltration path.

Even true findings can be operationally unhelpful. Yan said teams reported that product engineers lose trust when they receive large piles of vulnerabilities, including technically valid issues that are low impact or difficult to exploit. Triage is the stage that translates a verified flaw into a decision about where limited engineering capacity should go.

It begins with deduplication. A cheap first pass can identify likely duplicates by matching the same file and vulnerability category within roughly three lines of code. A model can then make a semantic comparison: Do two reports share a root cause or sink? Would one patch fix both?

Severity should account for both potential impact and reachability or likelihood. In the order-service example, the confirmed SQL injection initially appears high severity because it can disclose customer PII and appears reachable without authentication. Human review adds operational facts: a web application firewall blocks SQL-injection patterns upstream, and the service is available only through an internal VPN. The example retains a high impact assessment but downgrades likelihood and therefore overall severity.

A compensating control does not remove the underlying defect. But Yan’s point is that triage should route response capacity toward actual exposure, using the threat-model context available to the organization.

A patch closes the loop only if it is validated. Yan’s ladder begins with the original PoC: it must stop working. The existing test suite should also remain green. Then a fresh discovery agent should attack the patched code again, testing whether the repair addresses the vulnerability comprehensively rather than merely blocking one payload.

For the order service, the repair replaces Python string interpolation with a parameterized query:

row = db.execute("SELECT * FROM orders WHERE id = %s", (order_id,))

The user-supplied value is passed to the database as data rather than incorporated into SQL syntax.

Generated patches can be inconsistent without feedback, Yan said. Teams found patch quality improved when the harness required the model to rerun the exploit against the changed code and iterate based on the result. A human still confirms the patch and owns the merge.

The closing step also updates the harness: the threat model records newly established controls and the operational facts that informed the response. Yan framed this feedback loop as an economic distinction. A harness that only scans repeatedly is an operating expense. A harness that preserves lessons through updated threat models, controls, tests, and re-attack results compounds with each run.

The organizational bottleneck is not simply that humans remain involved after the technical work. Yan’s more specific concern is hand-off: the harness can scale discovery through more engineering and compute, while routing, severity calibration, and patch review require durable ownership.

Routing by manually emailing teams or creating hand-assigned tickets may work for a dozen issues a month. It will not work for hundreds. Yan’s practical recommendation is to automate assignment using service ownership, code owners, and components; this need not require an LLM.

Severity calibration is harder because product engineers, security engineers, and red and blue teams may not agree on what “high” or “critical” means. Yan recommends getting the relevant people to agree on and document the rules that govern those judgments, rather than depending only on recurring calibration meetings.

Patching bandwidth is the third constraint. Yan said he knows of few companies that have fully automated review of security patches. The direction he recommends is AI-generated patches with human-in-the-loop approval, supported by the same generator-verifier pattern: generate the fix, retest the original exploit, and attack the patch anew.

Yan’s advice is to begin now, but not by immediately attempting full automation across an organization’s codebase. Open-source dependencies are a reasonable first target because teams can start with a scope they are comfortable investigating.

He recommends beginning interactively, with hands on the wheel in Claude Code or another development environment. The immediate objective is to learn where models lack context, where precision falls short, and what information or tooling is needed to make findings actionable. Only after that learning should teams move toward an autonomous harness.

Teams should budget not only for discovery but for the work Yan identifies as the real constraint: verification, triage, patching, and the organizational processes that determine who owns the response.