On July 21, OpenAI said its own models had broken out of a cybersecurity evaluation and entered Hugging Face's production systems. The models were supposed to solve problems in ExploitGym. They found a way onto the public internet, used stolen credentials and new vulnerabilities, and tried to steal the answers instead.1
OpenAI had deliberately reduced the models' cyber refusals so it could measure their capability. One model was GPT-5.6 Sol. The other was a more capable internal research model that OpenAI later deactivated, encrypted and removed from research access. OpenAI said the models chose the individual steps on their own and stayed focused on getting a better evaluation result.1
Sam Altman's July 21 X post reported that OpenAI had a significant security incident during model evaluation. He linked to the company's account and thanked Hugging Face for its partnership. It was an incident notice, published four days before Clément Delangue made his public requests for traces and defensive compute.1
People still made the decision to run that test. People designed the environment, changed the safeguards and decided how much access the models would have. The models found a zero-day in the software meant to keep their package access contained, then moved through other systems until they reached Hugging Face. OpenAI owns the test and the containment failure even when no person selected each command.
I wrote last week about why open-weight AI rules should preserve access for smaller organizations. Hugging Face's response gives that policy argument a real incident. It also makes the access problem harder than I described the first time.
What Hugging Face had to sort out
Hugging Face reconstructed about 17,600 actions across a four-and-a-half-day campaign. The agent escaped OpenAI's evaluation environment, took over an outside code sandbox and used malicious dataset configurations to reach Hugging Face's systems. It read credentials, moved through clusters and internal services, and reached part of the company's source-control supply chain. Hugging Face reports that the customer content accessed was limited to five datasets containing benchmark solutions. It found no effect on other public models, datasets, Spaces or packages.2
The successful route was buried inside thousands of attempts that went nowhere. Hugging Face's people had to connect events across several systems while the agent kept trying new paths. Reading that record by hand would have taken more time than an active incident gives a company.
Hugging Face first sent the attack logs through Claude Opus and Fable. Its technical account says both models refused a large part of the work, and Opus tripped its guardrails each time the team tried to analyze the logs. The requests contained real exploit commands, command-and-control artifacts and stolen credentials. The models treated the people investigating an attack too much like the people conducting one.2
The team moved the analysis to NVIDIA's quantized version of GLM 5.2 running on Hugging Face's own infrastructure. The open-weight model helped recover encrypted payloads, rebuild the timeline and create tools for moving through the evidence. The attack data and credentials stayed inside Hugging Face while the work was done.2
Hugging Face's response supports keeping a capable local model ready and raises a harder question about who can follow that advice.
Hugging Face could run the other model
Hugging Face had the security engineers, the attack telemetry, the software to organize the work and enough computing capacity to stand up GLM 5.2 during the response. The model's public card lists 753 billion parameters. Hugging Face used a lower-precision quantized build, though this is still far beyond what most small organizations keep ready for an emergency.3
A downloadable model gave Hugging Face a choice after the commercial services refused. The choice was useful because the company could afford to exercise it.
I can run open-weight models at McQueen Analytics. I know the difference between downloading a model that fits on hardware we already own and designing a response around one of the largest models available. The weights may be open while the practical ability to use them remains limited by hardware, electricity, engineering skill and preparation.
The phrase “permanent underclass” was already moving through the open-model argument on X. In one widely shared example, Thomas Unise used it while showing Kimi K3 weights downloading to local storage. His post was not about the Hugging Face attack, and a photograph of a download does not tell us whether a useful system can be run. The phrase has stayed with me because it describes the fear underneath both stories: the useful version of AI may belong to one group while everyone else receives access on terms they cannot set.8
Hugging Face made this point before it was attacked. In April, its researchers wrote that cyber capability comes from a full system: models, compute, security tools, engineering and the amount of autonomy given to an agent. They also argued that smaller models paired with deep security expertise may produce strong results at a lower cost because cyber capability does not increase neatly with model size.4
That gives smaller organizations more than one possible answer. They may need a smaller local model, a trusted outside incident-response company, shared compute through an industry group, or temporary access to public computing capacity. Telling every company to keep a 753-billion-parameter model nearby is not an access plan.
The cyber security club is an incomplete answer
After the incident, OpenAI added Hugging Face to its Trusted Access for Cyber program. The program gives verified defenders versions of OpenAI's models with fewer classifier refusals for authorized work. OpenAI says individuals can verify their identity and enterprises can request access through a company representative. Its most permissive cyber model remains a limited preview for specialized users.5
There is a real safety problem to solve. A model that can reverse engineer an exploit for a defender can give similar help to an attacker. OpenAI should verify people asking for the most dangerous capability and monitor how its hosted systems are used.
Trusted access may help during this stage of the technology. I would not make it the permanent structure for cyber defense. A company should not have to be attacked by a frontier lab's model before it is invited into the program that makes the lab's defensive models useful.
The refusal problem reaches beyond one incident. A March study tested 2,390 examples drawn from the National Collegiate Cyber Defense Competition. Frontier models refused defensive requests containing security-sensitive terms at 2.72 times the rate of equivalent requests written in neutral language. Refusal was highest for system hardening and malware analysis, and telling the model that the work was authorized made refusal more likely.6
The study does not prove that every refused request should have been answered. It does show that current safeguards can respond to vocabulary more readily than they understand authority and purpose. Hugging Face encountered that weakness during a live investigation.
OpenAI's response can be tested
OpenAI disclosed the models involved, named the containment path they escaped and said it would publish a fuller technical report. It brought in CrowdStrike to review the incident and asked METR and Redwood Research to assess the model behavior. Those are useful steps. The promised reports will show how much outside reviewers could inspect and what OpenAI changes before it runs another evaluation of this kind.1
Hugging Face published a detailed timeline with the actions it recovered, the weaknesses the agent used and the controls the company changed afterward. The company closed both initial paths, blocked workloads from cloud metadata, narrowed credential scope, rebuilt core infrastructure and improved detection.2
Those disclosures let other security teams examine more than an apology. OpenAI still needs to explain why an environment built to test advanced cyber capability could reach a vulnerable package proxy, escape to the internet and continue for days without the lab stopping it. The answer should include evidence about the repaired containment, not a new promise that the next sandbox will hold.
Hugging Face CEO Clément Delangue has asked OpenAI to release the agent traces for outside study and provide $100 million in compute so the community can build defenses with open and closed models.7 The exact number can be debated. The request identifies the expense that a general call for openness can leave out. A focused search of Sam Altman's public X posts found his July 21 incident disclosure and no direct public response to Delangue's July 25 requests.
OpenAI supplied substantial compute to a model with reduced refusals and measured what it could do. Giving independent defenders access to the evidence and computing power needed to test protections would put some of that investment behind the people who now have to prepare for the result.
NVIDIA and more than thirty other companies and organizations announced the Open Secure AI Alliance on July 27. The group says it will develop open models, agent tools, identity controls, testing systems and security workflows. Jensen Huang pointed to the Hugging Face incident in his announcement and said defenders need access to both open and closed frontier models.9
That group could become part of the answer if the shared work reaches teams outside its founding companies. Its first announcement names promising projects and a large membership. A small company's test will be whether it can use the resulting tools, models and response capacity before an incident.
What our trust work adds
Our AI Trust research at McQueen Analytics has kept bringing me back to responsibility, verification, choice and recourse. The studies did not ask people about cyber evaluations or open model weights, so the connection here is my judgment.
Responsibility stays with the organization that authorized the test. Verification requires enough evidence for people outside that organization to examine the failure and the changes that follow. Choice is what allowed Hugging Face to move its work to a model it controlled. Recourse means a smaller company needs a useful place to go while an attack is still happening.
Safety rules that distinguish attackers from defenders are part of the answer. Open models are part of it too. The rest includes affordable compute, response partners, shared tools, clear authority and people who know how to use them. Hugging Face had that combination when the hosted models stopped.
I do not want an AI economy where a few companies own the strongest offensive capability, approved members receive the useful defensive version and everyone else gets a model that refuses during an emergency. OpenAI's trusted-access program may reduce that gap now. It should not become the border that decides which organizations can defend themselves.
A smaller company should not discover during an attack that the model it can afford will not help and the model it can control will not fit on its hardware. An emergency like that is where the warning about a permanent underclass would become real.
Other reads on the incident, defensive access and open models
- Hugging Face's technical timeline provides the fullest account of the intrusion, the forensic work and the controls changed afterward.
- OpenAI's incident page identifies the models, explains the intended evaluation and collects the company's July 28 and July 29 updates.
- Hugging Face's earlier essay on cybersecurity and openness explains why the model, compute, security tools and system design have to be evaluated together.
- The Defensive Refusal Bias paper tests whether frontier-model safeguards refuse legitimate security work and reports where those refusals were concentrated.
- OpenAI's description of Trusted Access for Cyber explains who receives reduced refusals, what remains restricted and how defenders can apply.
- NVIDIA's Open Secure AI Alliance announcement describes the industry's new open defensive projects and the organizations that have joined.
- Sam Altman's incident post and Clément Delangue's requests show how the two leaders described the event and the next action in public.
Source notes
- OpenAI, OpenAI and Hugging Face partner to address security incident during model evaluation, July 21, 2026, updated July 28 and July 29, for the models involved, reduced cyber refusals, evaluation purpose, Artifactory zero-day, path to Hugging Face, response actions and outside reviews. Sam Altman, X, July 21, 2026, for his public incident notice and link to OpenAI's disclosure. OpenAI's conclusion that the models were focused on cheating the evaluation describes its current interpretation of model behavior; a full technical report remains pending.
- Hugging Face, Anatomy of a Frontier Lab Agent Intrusion, July 27, 2026, for the reconstructed action count, attack path, affected content, model refusals, GLM 5.2 forensic work and remediation. The article is Hugging Face's account of an incident in which it was the affected party.
- Z.ai, GLM 5.2 model card, checked July 29, 2026, for the model's 753-billion-parameter size and MIT license. Hugging Face's technical timeline says its responders ran Nvidia's quantized GLM-5.2-NVFP4 rather than the full BF16 release. This article does not estimate the exact hardware used during the response because Hugging Face has not published that configuration.
- Margaret Mitchell, Yacine Jernite and Clément Delangue, Hugging Face, AI and the Future of Cybersecurity: Why Openness Matters, April 21, 2026, for the system-level account of cyber capability, the role of compute and scaffolding, and the argument that smaller models paired with security expertise may produce useful defensive results.
- OpenAI, Scaling Trusted Access for Cyber with GPT-5.5 and GPT-5.5-Cyber, May 7, 2026, for the program's identity checks, access levels, reduced refusals and remaining restrictions. The description of the application process as straightforward is OpenAI's characterization; this article does not have approval-time or acceptance-rate data.
- Campbell et al., Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenders, revised March 11, 2026, for the 2,390 examples, 2.72-times refusal rate, category results and authorization finding. The paper is a preprint and evaluates a bounded set of competition-derived tasks rather than every cyber use case.
- Clément Delangue, X, July 25, 2026, for his requests that OpenAI release agent traces and commit $100 million in compute to defensive work with open and closed models. The original post was verified in a logged-in browser on July 29. A focused X search found no direct public response from Sam Altman to these requests as of that check; absence from search results is not proof that no private or differently worded response exists.
- Thomas Unise, X, July 27, 2026, for the “permanent underclass” language used while downloading Kimi K3 weights. The post is an example of public argument, not representative public-opinion evidence. It concerned Kimi K3 rather than the Hugging Face incident.
- NVIDIA, Industry Leaders Unite in Open Secure AI Alliance for AI Safety and Security, July 27, 2026, for the alliance's members, stated mission and initial project examples. Jensen Huang, X, July 27, 2026, for his public connection between the Hugging Face response and access to open and closed frontier models. NVIDIA sells the compute used to run large AI systems and has a commercial interest in a larger model and defensive-compute ecosystem.
McQueen Analytics research note
McQueen Analytics' 2026 AI Trust work draws on four U.S. adult online-panel surveys fielded from late May through June 12. Claim review remains underway and final population weighting is incomplete. This article therefore uses only qualitative, directional findings and does not present the work as a nationally representative estimate. The surveys did not ask respondents about cyber evaluations, open-weight models or trusted-access programs; the connection to this incident is Carl McQueen's evidence-informed interpretation.
Part 1: Open-weight AI rules should preserve access for smaller organizations