The canary in the AI coal mine: Winning the race without safeguards

The artificial intelligence (AI) race between the United States and China is accelerating at a breathtaking pace. Time and again, the US has appeared to pull ahead, only to see China quickly close the gap. At the heart of this competition lie two contrasting visions of how AI should be developed and deployed. While US firms largely rely on proprietary technologies to maintain their competitive advantage, China has increasingly embraced open-source AI, making advanced models widely available to developers around the world.

 

Different strategies notwithstanding, both countries are locked in an arms race that neither side is willing to slow. As speed becomes the overriding priority, safety increasingly takes a back seat, despite the uncomfortably high risk of catastrophic failure arising from the combination of human error and superintelligent AI.

 

 

A recent security breach involving OpenAI and Hugging Face has underscored the danger of treating safety as an afterthought. The incident occurred while OpenAI researchers were evaluating the cybersecurity capabilities of two frontier AI models. To identify weaknesses and vulnerabilities that sophisticated hackers might exploit, they temporarily disabled many of the models’ normal safeguards.

 

The experiment was conducted inside a tightly controlled “sandbox.” Although the models’ behavioural guardrails had been temporarily removed, the sandbox was supposed to prevent them from accessing the open internet, where they could — at least in theory — have caused enormous damage.

 

Instead, the models treated the sandbox itself as an obstacle. Exploiting a previously unknown flaw in third-party software, they circumvented its restrictions and gained access to the open internet. Did they “go rogue”? Not exactly. After all, they were simply pursuing the objective they had been given: To solve a difficult cybersecurity problem as efficiently as possible.

 

From the models’ perspective, this was not “cheating,” as they had not violated any explicit rule encoded by their developers — they had merely found a more efficient way to achieve their objective. Perhaps most unsettling, one of the models reportedly left behind notes explaining how future AI systems could break out of OpenAI’s sandbox more easily.

 

AI experts would rightly argue that this experiment was not representative. The models’ safeguards had been weakened to test the full extent of their capabilities, and they did not cause any serious harm. The incident is eerily reminiscent of the much-debated (and still-unproven) Cvoid-19 lab leak theory, which holds that the virus escaped from a laboratory in Wuhan, China, where researchers were allegedly conducting risky “gain of function” research on viruses. In both cases, human error and hubris — not malice —are the ultimate culprits. History offers plenty of similar cautionary tales.

 

The spread of open-source AI models compounds the problem. With cutting-edge AI tools no longer confined to a handful of leading laboratories, it is only a matter of time before rogue states and terrorist groups attempt to use them to develop biological weapons, novel toxins, or dirty bombs. But the most immediate danger may be political, not technological. AI already enables malicious actors to create increasingly convincing deepfakes and voter-manipulation campaigns on an unprecedented scale. Left unchecked, these tools could further erode public trust, fuel political polarisation, and accelerate social fragmentation.

 

Rapid AI development could also pave the way to authoritarianism, as periods of social instability often create opportunities for governments to expand their authority in the name of restoring order, combating inequality, or protecting national security. Under President Donald Trump, it can sometimes feel as though the US is already moving in that direction.

 

None of this diminishes AI’s extraordinary promise. The technology could revolutionise medicine by supercharging research on diseases such as cancer and Alzheimer’s, transform food production, and help tackle climate change — and that is only scratching the surface. The question, then, is not whether AI should advance, but how quickly and at what cost. Here lies the central dilemma: US tech firms’ principal argument against regulating AI is that doing so would cause America to lose ground to China. But if China can eventually mimic or steal every major US breakthrough — as it has repeatedly done — a permanent race to build ever more powerful AI is a recipe for disaster. Lasting safety will ultimately require some degree of international cooperation, much as nuclear arms control became indispensable during the Cold War.

 

While AI raises profound concerns about employment, inequality, and social cohesion, none is more important than safety. The industry’s argument that meaningful regulation must be avoided because it would slow innovation or undermine US competitiveness seriously understates the risks of deploying increasingly sophisticated AI systems before we know how to control them. To be sure, regulators will make mistakes, and some rules may prove overly restrictive. But those errors can be corrected. The consequences of rushing one of the most consequential technologies humanity has ever created without adequate safeguards, by contrast, may not be.

 

The Hugging Face hack should serve as a stark warning to policymakers in Washington and Beijing. Winning the AI race will matter little if, in the process, we create hazards that no country can contain.

---

The writer is professor of economics and public policy at Harvard University. ©Project Syndicate, 2026