By ASF Tooling and ASF Security
During a three-day window in August 2026, the Apache Software Foundation (ASF) ran full security scans across 230 of the foundation’s repositories. The work was done by the ASF Security and ASF Tooling teams, within the Foundation’s Responsible AI Initiative (RAI), using Anthropic’s Claude Mythos 5 through their Project Glasswing security program. Findings from scan results are now being shared with the projects that own the code through the Foundation’s official disclosure path, and remediation is underway.
This post describes how that came together: what prompted the scans, what we had already built, what the projects did to prepare, how the scans ran, and where the results went. It is a report on work in progress rather than a finished story, made possible by a recent donation by Anthropic.
Why we started
Two events happened close enough together to become one effort. The first was that ASF Security received Mythos-based scans run by friendly parties from Alpha-Omega on two ASF projects and analyzed the results. Automated security analysis is not new to the Foundation, but these were different: specific, well-reasoned, and detailed enough to provide value. Seeing what the method could do on real ASF code changed the question from whether to use it to how to use it well.
The second event is the growing volume of AI-generated vulnerability reports arriving at open source projects. Anyone can now point a capable model at a codebase and forward whatever results come out. Some of those reports are helpful. Many are not, and separating the two comes at the cost of maintainer attention that projects often do not have to spare. Even reviewing constructive reports could take up too much time. The problem is going to get worse, regardless of what the Foundation does.
Together these events pointed at the same potential solution. If AI-generated security analysis is going to arrive at ASF projects, the Foundation should be the one running it with the projects, in terms of the projects set, and with enough context that the output is worth reading. This motivation generated the threat-modeling effort the Security team would implement.
Meanwhile, ASF Tooling had been working on the same problem from the other side. Scanning a project on demand does not scale past a handful of projects and leaves everyone else waiting. Tooling wanted scanning to be something a project could simply have available, alongside other Foundation-provided infrastructure.
What ASF Tooling had already built
Since early 2026, ASF Tooling has been running an automated audit pipeline that evaluates code against the OWASP Application Security Verification Standard (ASVS). It runs on Gofannon, an agent platform the Tooling team has been developing.
The original goal was code quality on Apache Trusted Releases, the release-management platform Tooling is building: a specific codebase, a specific standard, a specific team that wanted to know where it stood. In developing an automated security scanning pipeline for its own use, Tooling found it had built a system general enough to apply to other codebases across the Foundation, and the purpose widened to providing scanning as a managed service any ASF project could use.
The pipeline is built around three model tiers: A light tier does high-volume filtering; a medium tier builds inventories of what the code contains; and a heavy tier does the analysis that needs real reasoning. The choice of model per tier, and parameters applied for each task, are configurable at runtime. The pipeline runs on an Opus, Sonnet, and Haiku ensemble, or on a Mythos, Gemma, and Qwen ensemble that puts self-hosted models on the two lighter tiers. Swapping between them does not touch the pipeline itself. That flexibility allows a balance among quality, speed, and cost for any given use case.
This process is automated: a project sets what it wants scanned, and the run proceeds, from a browser or through an API without supervision.
Through development of this audit pipeline, we discovered the critical need to provide LLMs with contextual information about the codebase they are reviewing regarding security posture, architectural decisions, team code standards, and even private information about deployment choices. These audit guidance inputs drastically reduced false positives — and even completely wrong inferences — from surviving through to a final report.
When we began the work with Project Glasswing, the pipeline found a second use. Running both approaches over the same repositories and comparing what each produced, turned out to be a good way to learn what each approach is actually good at which fed into how we tuned the Glasswing scans.
Preparing the projects
To set up scanning for success, ASF Security invited interested projects to work through a threat model. 75 Project Management Committees (PMCs), with more than 180 repositories, signed up. Each produced a description of its own security posture: the components that matter, the boundaries that are supposed to hold, and the assumptions the project has already made and does not want relitigated. In other words, what is fully trusted by design, what is delegated to whoever operates the software, and what is explicitly out of scope. A Threat Model skill contributed by Alpha-Omega enabled the Security Team to create individual threat models using an initial template, and helped them maintain ongoing communications with the PMCs at a time to create new models as well as verify and complete existing ones.
ASF Security reviewed every threat model before it fed anything downstream, checking that the model was correct and that it actually applied to the code as written.
The preparation paid for itself directly. Scanning against a reviewed threat model cost roughly one-fifth less than scanning without one, because the analysis spent its effort on the parts of the system the project considers most consequential instead of rediscovering documented design decisions. And even better, saving time of the maintainers, it allowed them to limit the findings to those that do not fall outside of the model.
Glasswing and Mythos 5
Anthropic’s Project Glasswing program has given a number of organizations access to the company’s most capable models for security research. The ASF joined as a participant to support the work.
Each scan is anchored on the project’s threat model. Scans were performed with Glasswing’s harness running in parallel sessions of Claude Code against Mythos 5. We did not customize the harness and provided a series of simple prompts to kick things off.
The quality of findings held up with low levels of false positives along with critical vulnerabilities each packaged with a drafted, separately reviewed patch attached. Getting to that level of results took minimal iteration, and in two batches we had our full result set.
Disclosure and remediation
Everything from these scans runs through the Foundation’s official disclosure path, flowing through Security governance and landing with responsible parties at each project.
Sensitive findings go to the affected project’s security or private list first. The PMC assesses the finding, decides what it means for their code, and remediates on their own timeline, as they would for a report from any other source.
Reports are ordered by what a maintainer should deal with soonest given how the software is actually deployed, rather than by severity label alone. A high-severity finding in a code path nobody reaches is not the first thing a volunteer should spend a Saturday on, and the reports say so.
What’s next
We are planning three enhancements.
First, we will make scans incremental. A repository scanned once should not be scanned from scratch the next time. The goal is a scan that knows its prior results, tracks what has changed in the threat model, and follows what has actually been fixed and what has been published as a CVE. We identified opportunities to avoid findings covered by the threat model by doing additional adversarial reviews. We also treat the first batch of reports as a learning opportunity and gather feedback from the PMCs that received them.
Additionally, we are looking at other scan types. Review of incoming vulnerability reports are the most immediately useful, given where this project started. Beyond that, we will implement additional OWASP specifications and other standard scan types, each looking at code from a different angle.
Finally, we are continuing to build toward scanning as a self-serve function, where a project chooses the scans it wants and runs them against a token budget the Foundation manages, rather than waiting for a small group to run them manually.
ASF Security (Mark Cox, Jarek Potiuk) and ASF Tooling (Dave Fisher, Andrew Musselman) would like to thank Anthropic (Jake Richards, Karlo Anton) for the Glasswing collaboration and the donated capacity that made this work possible, and the 75 PMCs who did the threat-modeling work that made the results worth reading (special thanks: Evan Rusackas, Piotr Karwasz, Jan Friedrich, Sean Palmer, Alastair McFarlane, Emmanuel Lécharny, Adam Monsen, Greg Stein, Paul King, and James Fredley).
