Microsoft AI chief says Claude training risks making AI harder to control

Microsoft AI chief Mustafa Suleyman said he shared Anthropic's focus on safely managing AI, but flagged risks in the way it trains its Claude chatbot on ideas related ‌to consciousness and welfare interests. 

Suleyman called ​for removing all speculation ​about consciousness from AI training documents, arguing such language could undermine humanity's ability ​to control superintelligent systems.

 "We're all focused on the same aim, which is to try to control a superintelligence," Suleyman told Reuters in an interview on Tuesday. "I think that's going to be the greatest challenge that we face ​in the 21st century."

 Suleyman said teaching Claude that it might deserve welfare would "make ‌it a lot harder to turn it off or to ​control it."

 

 The dispute comes as AI safety concerns mount, with Anthropic CEO Dario Amodei calling for a slower pace of frontier-model development to allow safeguards to catch up, ‌and OpenAI CEO Sam Altman ​and Elon Musk also urging ‌greater caution around the most powerful systems.

 Suleyman, in an essay on Wednesday, ‌acknowledged Anthropic's "seriousness and good faith," calling Amodei and his team thoughtful and ​principled researchers who genuinely care about humanity's future.

 Suleyman said Anthropic made a mistake by embedding speculation about consciousness in Claude's ​training materials, arguing that the model's statements about possible feelings or moral status cannot be treated as independent evidence because its training ‌encourages such reflections.

 "I think they have good intentions, and they really ‌are trying to work towards safety. But I think that they have made a mistake," he said. "They're not emerging naturally. They're emerging as a result of the training regime."