Authors:
Arie Haenel, INT31
Tony Martin, INT31
AI is rapidly changing how security researchers search for vulnerabilities, creating the potential to examine more code, uncover subtle patterns, and strengthen the hardware and software foundations on which AI itself depends. But raw model capability is only the starting point. The real test is whether AI can produce precise, evidence-backed findings that security teams can trust and developers can act on.
That challenge was at the heart of Project Glasswing, a collaboration between Anthropic and critical-infrastructure companies that gave participating researchers early, time-boxed access to the frontier model Mythos. Intel’s INT31 team used the opportunity to scan hundreds of repositories spanning firmware, drivers, cryptographic libraries, and cloud container images. What followed was more than an experiment in prompting. It became a practical lesson in how grounded context, deterministic tools, human expertise, and carefully designed workflows can turn frontier AI into a more effective vulnerability-hunting system.
Point a frontier model at a codebase and tell it to "find the vulnerabilities," and it will usually produce dozens of findings, but many will be of little to no value. A naive scan does not solve the needle-in-a-haystack problem: determining what is exploitable. It just moves the needle into a larger haystack of plausible-sounding false positives. A handful of genuine findings buried among a thousand candidates can be operationally equivalent to finding nothing. If a tool cries wolf too many times, people stop listening, and its real findings die alongside the false ones. This problem grows worse lower in the stack where software is closer to the hardware. Low-level systems have many complex entry points and the threat actors and trust boundaries are more complicated; these details cannot always be understood from the code. Findings become harder to verify automatically, requiring costly human investigation. If every finding must be investigated, noisy results burn the two things a security team can't get back: time and trust.
The Real Goal Is Precision
This was not an exercise to "find the most bugs." Our goal was to produce a more trustworthy set of actionable findings. Simply put, precision. To be clear, this does not mean scanning less thoroughly. We did not want to knowingly miss vulnerabilities; thoroughness still matters. We optimized for usable precision to minimize the impact on a fixed supply of expert attention.
We separated our findings into one of three outcomes:
| Term | Meaning |
|---|---|
| True Positive | A reported vulnerability supported under the engagement's stated adjudication or verification standard. |
| Weakness | A real defect or risky condition for which in-scope exploitability has not been established. |
| False Positive | A vulnerability claim contradicted by evidence: absent from the target build, impossible under validated assumptions, or otherwise incorrect. |
Weaknesses are where large language models shine; they are very good at finding coding flaws but this can be misleading. Establishing exploitability is much harder, especially lower in the stack, closer to the hardware where most code is in C and C++ where conditional compilation, build flags, and complex call paths change what code exists in an image. These impact whether an attacker can reach a finding. A tool that reports weaknesses as if they were exploitable vulnerabilities generates excessive triage work. Weaknesses are still important from a defense in depth perspective, especially in code paths that process untrusted data, but finding exploitable issues is the first priority.
It Is Not A Prompt Problem
When the output is noisy, the instinct is to write a better prompt: add a "be careful", a few examples, a sterner system message. While strong prompting helps, its impact has a ceiling. Better prompts cannot recover missing code, absent threat context, or the difference between the source code and what was in the compiled image.
The largest gains we saw came from enhancing the system around the model, such as grounded inputs, reproducible tools, deterministic validation, and breaking a problem into manageable parts, not the prompts you sent. The lower the code sits in the stack, the more critical a rigorous system becomes to producing high-quality results.
Minimize The Stochastic Surface; Maximize Determinism.
By their nature, large language models bring randomness. So, we concentrate open-ended reasoning into tightly controlled sections that are surrounded by deterministic and reproducible functions such as input preparation, tool execution, and output validation. Every uncontrolled stochastic step can increase variance and creates another opportunity for an unsupported claim.
In practice, the system became a multi-stage harness that deliberately separates context preparation, discovery, and verification. We leveraged deterministic work such as scripts, libraries, and parsers to prepare the context and to disprove candidate findings, while reserving expensive, probabilistic reasoning for the narrow slices that genuinely need it.
Feed It The Truth, Not A Guess
The better the context provided to a model, the better the results. Perfect context is impossible, thus there are always gaps. A model reasoning on a section of code with missing context becomes a false-positive factory.
The Binary Is Ground Truth
In C/C++, not all the source code ends up in the binary: preprocessor conditions (#ifdef and similar), build flags, template instantiation, linker selection, link-time optimization, and optimizer decisions all change what is included. An optimizer can even remove an ordinary memory-clearing write through dead-store elimination when it concludes the write has no observable effect; if that write is security-relevant, its removal may itself create a vulnerability. Large language models struggle to follow these interacting mechanisms. Debug information in a compiled binary can prove whether a reported line of code was included in the build, providing a zero token cost dismissal of findings in dead or conditionally excluded code; missing line mappings alone do not prove absence.
Retrieval Locates Evidence; It Must Never Write It.
One of the risks when using AI to analyze security context is the generation of plausible-sounding details that are simply not true: hallucinations. To prevent this, we have a rule, the model may point to a location to retrieve evidence, but it should not be trusted to cite or quote evidence.
When reviewing code for security issues, many apparent problems turn out to be intentional design choices. Those intentions are usually buried in a design document, diagram, or a planning spreadsheet, rather than in the code itself. A model summarizing those documents from memory can subtly rephrase or misremember the content, changing the meaning in ways that are hard to catch.
Our approach is to have the model return candidate locations of information with provenance: a specific file, page in a document or a row in a spreadsheet. The harness then reopens the original source and supplies that exact material to the next step in the process. Generated summaries can help navigation, but they are not treated as evidence.
An early version of our system produced a report containing realistic but fabricated function signatures, code the model had invented. The fix was not a sterner prompt. It is an architecture where every quotation in a report is checked mechanically against the original source under a defined normalization policy, and if the text does not match, the section of the report is rejected or flagged. A matching quote proves only that the citation is accurate: the model quoted the source faithfully. It does not prove the document is current, that the model interpreted it correctly, or that the design it describes is truly safe.
Grounding is a necessary system control.
Prove The Model Wrong, Cheaply
The model is not the only decision maker in a harness at various steps, and often it should not be the one making it. When a security-relevant property can be represented precisely, we use an appropriate analyzer rather than asking a model to guess. Static reachability and data-flow tools provide repeatable evidence within their analysis assumptions. Using a solver to mechanically check whether a precisely stated logical constraint is satisfied can give an exact result, but that result is only as good as the encoding, the translation of the code's real behavior into that constraint. The workflow is not "AI replaces static analysis"; it is hypothesis generation followed by deterministic, adversarial verification, and these repeatable checks reduce false positives.
Here is what one rejection looks like. A pattern scan flagged two nearly identical functions in a crypto library, both computing a length from a signed integer multiply, where an overflow results in undefined behavior before parsing attacker-supplied bytes. The difference is not visible in the local pattern; it is where the operands come from. In the first case, the inputs come through the library's own APIs, so the overflow cannot be ruled out, and we keep it for deeper analysis. In the second, the operands are read from validated state and constrained by hardware to a small range, so the values cannot overflow, and the candidate is dropped before any expensive reasoning runs. An incorrect but convincing candidate can be rejected cheaply.
When the model spots a genuine pattern rather than a one-off, we turn it into a static-analysis rule and scan the rest of the codebase at far lower marginal cost. The rule provides repeatable coverage. If it finds new candidates, these are not yet confirmed vulnerabilities and still require contextual adjudication. The stochastic tool seeds the deterministic.
Control What Reaches The Frontier Model
The single biggest costly mistake is running the most powerful model for every task. In our measured Glasswing workloads, frontier-tier calls dominated token spend. The entire optimization game is controlling what models are used in the various stages and functions. We reserve frontier reasoning for complex, unfamiliar code paths and push routine preparation and triage to scripts and less expensive models working on a tight, pre-digested slice of context.
By routing only the hard cases to frontier reasoning and reusing cached context from supporting documents instead of regenerating it each run, we materially reduced our token spend, concentrating it in the analysis that genuinely needed deep reasoning. Tokens are only part of the total cost, false positives cost researcher time and drive reviewer fatigue; a scan that costs slightly more but is substantially more precise is often cheaper overall.
The Takeaway
One design principle organized everything else: if you ask a model to do too much, it will do less.
- Build a system, not a prompt and minimize the stochastic surfaces: Concentrate open-ended reasoning where needed and keep everything around it deterministic and reproducible.
- Feed the model the truth: For a given build, use the final artifact as the authority for what code and properties are present. Use retrieval to locate evidence rather than trusting the model's recall.
- Prove the model wrong, cheaply: For well-defined and bounded questions, deterministic verification is more valuable than probabilistic reasoning and should be used before paying for deep reasoning.
- Use frontier models where they are needed: Route genuinely hard, bounded problems to expensive models, and use less costly models or exact tools where they can satisfy the various tasks.
None of these four points are about a clever prompt. It is about a system designed to reduce unsupported claims and increase precision.
In our experience, a well-designed harness with a mid-range model can yield higher usable precision and lower human triage effort than a poorly designed, single-shot, frontier only workflow. This gap between the approaches tended to widen lower in the stack. Frontier models are remarkable, but the engineering around them is what turns raw capability into a trustworthy set of findings for a development team.
Share Your Feedback
We want to hear from you. Send comments, questions, and feedback to the INT31 team.
About the Author
Tony Martin is a Sr. Principal Engineer in Intel’s INT31 Security Research team, where he focuses on emerging threats, software architecture and AI security. He has discovered thirty CVEs and one CWE and is senior staff at DEF CON’s Packet Hacking Village.
Arie Haenel is a Principal Engineer at Intel, where he leads ASSERT, an Offensive Security Research team. He has over 25 years of professional experience, in security research and security product development on a vast number of embedded platforms, at Intel, Cisco and NDS. In his spare time, Arie teaches security engineering as an Adjunct Lecturer at the Lev Academic Center.