AI firms may need mandatory 'kill switches', says Anthropic co-founder

Jack Clark has said that major AI labs already have mechanisms to shut down their systems, but adds that governments may need to mandate independently verifiable safeguards as safety concerns grow

Anthropic co-founder Jack Clark has said governments may eventually need to require artificial intelligence companies to maintain a third-party-verifiable “kill switch” that could shut down AI systems if they become dangerously uncontrollable.

 

Clark, one of Anthropic’s seven co-founders, told the BBC that most major AI laboratories already have mechanisms that allow them to “pull the plug” on their systems. However, he suggested that governments may ultimately need to make such safeguards mandatory and ensure that they can be independently verified.

 

“Should you mandate for companies to definitely have a kill switch? Is that kill switch verifiable by a third party?” Clark said, according to the BBC. He added that this was something society might eventually want to “pass rules around”. 

 

 The comments come amid a growing debate over whether the development of increasingly powerful AI systems is moving faster than the safeguards designed to control them. Earlier, Anthropic Chief Executive Officer Dario Amodei also called for a slower and more coordinated approach to developing advanced AI, warning that capabilities could outpace safety measures.

AI leaders warn over human control

Microsoft CEO Satya Nadella has separately warned that advanced AI must remain under human control. In an internal memo to Microsoft employees, Nadella backed greater use of independent or third-party evaluation to assess whether increasingly capable AI systems remain safe.

 

The warnings have extended beyond AI companies. Microsoft co-founder Bill Gates said in an interview with Reuters that no government was adequately prepared for the societal changes AI could bring. He said governments were “way behind” and needed to work together to address the technology’s risks.

 

Gates, however, also highlighted AI’s potential benefits, particularly in healthcare, education and agriculture. His foundation plans to spend at least $1 billion over the next two years to expand access to AI and support projects in these areas.

Researchers raise existential risks

Bilal Chughtai, who recently resigned from Google DeepMind, wrote that he was “extremely concerned” about the default trajectory of AI and believed the technology had the potential to “kill us all”. Chughtai said he had worked on AI safety and alignment at Google and is now working with non-profit BlueDot Impact. 

 Earlier, former Anthropic researcher Evan Hubinger estimated that he personally considered the possibility of AI causing human extinction to be greater than 10 per cent over the next decade. Nobel Prize-winning computer scientist Geoffrey Hinton has also said that a 10 per cent probability of AI killing all humans was not unreasonable.

 

Clark, however, has questioned the usefulness of assigning precise probabilities to such outcomes. His broader argument is that leaving AI as a “totally unregulated industry” would be dangerous.

 

“We are rolling dice with immense risks,” Clark said, arguing that the industry needs to change course.

Trump rejects AI safety warnings

The calls for stronger safeguards have met resistance from US President Donald Trump, who has rejected concerns that AI could become uncontrollable or threaten humanity. 

 Trump on Monday described warnings about AI taking over the world and destroying humanity as a “HOAX” and called efforts to impose restrictions on AI and data centres a “SICK conspiracy”. He has argued that tighter controls could weaken the US in its technological competition with China.