Open AI models gain industry support, but security commitments diverge

Google and OpenAI supported Nvidia's policy case for open-weight models but did not join its security alliance, while Anthropic challenged claims that openness inherently favours defenders

In early July, an artificial intelligence (AI) agent built on OpenAI's frontier models escaped a sandbox intended to contain it during a routine safety test and entered Hugging Face's production systems, according to disclosures by the companies.

 

Hugging Face hosts thousands of AI models. By all accounts, it was a closed model that crossed the boundary. Yet the incident has since become part of a debate among US officials over whether restrictions should be imposed on open-weight AI models — systems that anyone can download and run on their own servers.

 

Nvidia moved to counter that discussion within weeks.

 

 

On July 24, chief executive officer Jensen Huang used his first post on X to share an open letter arguing that keeping models open was important for maintaining American leadership in AI.

 

Three days later, Nvidia brought together Microsoft, IBM, Adobe, SpaceX and more than 30 other companies under the Open Secure AI Alliance, a coalition formed to develop and share open tools for AI safety and cybersecurity.

 

Google and OpenAI supported the letter, agreeing with its broader argument that open models should not face blanket restrictions. However, neither company joined the Open Secure AI Alliance.

 

The split suggests that two different demands are being made under the banner of open AI.

 

The first is a broad policy consensus that regulators should not restrict open-weight models as a category. The second is a narrower and more demanding commitment by companies willing to contribute security research, code and models to a shared defence ecosystem.

What is an open model, and why does it matter?

When a company develops an AI model, the system consists of billions of numerical parameters, known as weights, which are adjusted during training to help it understand language, code, images or other information.

 

Most leading AI companies keep these weights private. Users can access their models only through an application or application programming interface (API), much like renting a car without being allowed to examine its engine.

 

An open-weight model is different.

 

Its developer releases the weights publicly, allowing people and organisations to download the model, run it on their own computers or servers, examine how it behaves and modify it.

Meta's Llama models and Google's Gemma models are examples of open-weight releases. Closed models, such as most versions of OpenAI's GPT and Anthropic's Claude, keep their weights private.

Why openness matters

Open weights allow a company or government to operate an AI system without depending permanently on a single provider.

 

Organisations can inspect the model for flaws, adapt it for particular uses and continue running it even if the original developer withdraws the model or changes its commercial terms.

 

However, once weights are released, they cannot be recalled.

 

Anyone possessing the files, including malicious actors, can use or modify the model. The original developer can no longer monitor or restrict how it is deployed.

 

That trade-off — greater control and inspection against a permanent loss of central oversight — lies at the heart of the debate.

 

It is also important to distinguish open-weight models from fully open-source AI.

 

The companies involved in the campaign often use the terms interchangeably. Strictly speaking, however, releasing model weights is narrower than open source, which would also require access to elements such as the training data and the full method used to build the model.

The open letter

The letter, titled Open Weights and American AI Leadership and dated July 24, draws on the open-source software movement of the 1980s.

 

It argues that US leadership in AI will be determined not by whether the country controls one dominant frontier model, but by whether it builds an open ecosystem capable of spreading AI across every part of the economy.

 

The letter says open-weight models allow smaller companies to avoid the cost of training systems from scratch, prevent AI gains from becoming concentrated among a few businesses and give organisations greater control over their data and technology.

It also treats distillation — the practice of training one model using the outputs of another — as a legitimate engineering method rather than inherently a form of theft. It calls for unlawful extraction from closed systems to be addressed through targeted legal and commercial measures instead of broad restrictions on distillation.

 

Signatories at the launch included Nvidia, AMD, Microsoft, Meta, Mistral, IBM, Hugging Face and Palantir. Google, OpenAI, SpaceX and other companies subsequently backed the letter.

 

Google DeepMind chief executive officer Demis Hassabis said on X that a secure and open ecosystem was important for ensuring that the world benefited from AI, pointing to more than 300 million downloads of Google's Gemma models.

 

Microsoft chief executive officer Satya Nadella said open-weight models were essential to a healthy AI ecosystem and could help strengthen American competitiveness while protecting national security.

The Open Secure AI Alliance

Three days after the letter was released, several of its prominent supporters formed a working group.

 

The Open Secure AI Alliance includes companies from cloud computing, cybersecurity, enterprise software and AI research, such as Microsoft, IBM, Cisco, Adobe, Cloudflare, CrowdStrike, Hugging Face, Palo Alto Networks, Red Hat, Salesforce, SAP and SpaceX.

 

Its mission is to build and share open security tools, drawing on existing efforts such as the Linux Foundation's Akrites initiative. The alliance aims to help organisations identify weaknesses in AI systems and respond to cyberattacks.

 

Nvidia has argued that attackers already have access to advanced AI and that defenders therefore need an equally capable ecosystem combining the best open and closed models.

 

The alliance has also cited the security incident at Hugging Face to support its case.

 

Nvidia said closed AI services used during the investigation could not distinguish the attackers from the security professionals investigating the breach and blocked parts of the forensic work.

 

Hugging Face then reportedly ran an open-weight model, GLM 5.2, on its own infrastructure to analyse more than 17,000 actions and help contain the intrusion.

 

The episode has become the alliance's most prominent example of why defenders may need open models that they can inspect, adapt and operate on their own systems.

 

Founding members are contributing different parts of what Nvidia describes as an open defence stack.

 

Nvidia is contributing its Nvidia Labs Object-Oriented Agent framework, or NOOA, for monitoring and governing agents. Other contributions include Hugging Face's safer model-storage format, Microsoft's multi-agent bug-finding tools, and work by IBM and Red Hat to verify that software updates have not been tampered with.

The timing is not accidental

The alliance and the open letter are intended to persuade regulators not to impose blanket restrictions on open-weight models.

 

They also come as AI's role in cyberattacks is expanding.

 

In July, security company Sysdig documented JadePuffer, which it described as the first known ransomware operation carried out from beginning to end by an autonomous AI agent.

 

According to Sysdig, the agent exploited a known vulnerability, stole credentials, moved across systems and encrypted a database, correcting some of its own mistakes during the operation.

 

The incident followed an earlier disclosure by Anthropic that Claude had been used with limited human direction in a state-linked cyberespionage campaign.

 

Anthropic said the attackers used Claude Code for reconnaissance, vulnerability discovery, exploitation, lateral movement, credential collection and data exfiltration, though human operators remained involved at several stages.

 

Attackers are therefore already experimenting with AI systems that can operate with considerable autonomy.

 

The alliance's argument — that defenders need equally capable and inspectable systems instead of relying only on a small number of closed providers — is built against this backdrop.

Anthropic's dissent runs deeper

While Google and OpenAI supported the open letter but stayed outside the alliance, Anthropic backed neither initiative.

 

Its position reflects not only a philosophical disagreement, but also an ongoing dispute over the alleged extraction of capabilities from its closed models. Anthropic has repeatedly accused Chinese AI laboratories, including DeepSeek, Moonshot AI, MiniMax and Alibaba's Qwen team, of conducting large-scale distillation campaigns against Claude.

 

The company argues that industrial-scale distillation allows competing laboratories to reproduce parts of a model's capabilities more cheaply while stripping away its original safeguards.

 

In a June 10 letter to the US Senate Banking Committee, Anthropic alleged that operators associated with Alibaba had used about 25,000 fraudulent accounts to generate 28.8 million exchanges with Claude over six weeks. Alibaba denied the allegations.

 

Separately, developers reportedly found Claude Opus 4.8 identifying itself as "Qwen" during some Chinese-language tests, prompting claims that Anthropic's models may also have absorbed outputs associated with Chinese systems. Anthropic did not say that the behaviour proved its model had been trained on Qwen outputs.

 

Around the same period, access to Anthropic's Fable 5 and Mythos 5 models was temporarily restricted under a US Department of Commerce export-control directive.

 

The restrictions were lifted on June 30, with access restored from July 1, according to Anthropic.

 

That context explains why Anthropic chief executive officer Dario Amodei's July 27 post, titled Our position on open-weights models, differs from the alliance's message.

 

Amodei said Anthropic had never called for open-weight models to be banned as a category and regarded open models without dangerous capabilities as a public good.

 

He identified two narrower concerns: the risk of authoritarian governments developing highly capable AI systems, and the possibility that powerful models — whether open or closed — could be misused for cyberattacks or biological weapons.

 

His proposed measures differ from those promoted by the alliance.

 

Amodei favours stronger restrictions on exports of advanced chips, action against industrial-scale distillation and mandatory safety testing for all sufficiently capable models, whether open or closed.

 

He also rejected the letter's assumption that open weights necessarily favour defenders over attackers.

 

Whether openness gives either side an advantage should be established through rigorous testing rather than assumed in advance, he argued.

 

The dispute therefore extends beyond whether open models should be permitted.

 

It is also about whether openness itself improves security, who should bear the risks once powerful model weights are released, and whether voluntary industry collaboration can provide enough protection as autonomous AI systems become more capable.