Doomsday conversations snowball at AI firms as workers urge caution

Anthropic researcher Jacob Coxon's resignation has amplified concerns across leading AI labs, with some workers urging executives to slow development until stronger safeguards are in place

By Mike Isaac & Kate Conger

 

When Jacob Coxon, an Anthropic researcher, publicly quit the artificial intelligence (AI) start-up this week, citing his concerns over the powerful technology, the internal messages began flying.

 

“It’s what we’ve all been saying,” one Anthropic employee wrote about Coxon’s comments, in a group chat among workers that was viewed by The New York Times. “It’s just public-facing now.”

 

“Jacob took the group chat public,” another employee wrote. “That’s a good thing.”

 

The reactions inside Anthropic were echoed by AI researchers at OpenAI, Meta and Google, who parsed Coxon’s social media posts that described how OpenAI and Anthropic were “gambling with our lives” with the technology.

 

 

For those who have made AI their life’s work, Coxon’s message was not new. But his comments snowballed into such a global conversation that many saw it as a moment to call for increased caution. Some AI workers are now speaking up to top executives to urge a pause in developing the technology until it has the proper safeguards, six of the people said.

 

The conversations underline the tensions within the leading AI labs over the safest ways to develop the technology. Even as employees become emboldened to call for a slowdown in AI development, companies like Anthropic and OpenAI are juggling priorities.

 

Both are planning blockbuster initial public offerings (IPOs) soon. The companies are also fearful they could face antitrust concerns if they agreed among themselves to take a breather in AI development, two people with knowledge of the matter said. And Chinese start-ups have produced competitive AI models.

 

That has created a tricky situation for top AI executives, many of whom agree that the technology poses risks. Sam Altman, the chief executive of OpenAI, wrote in a 2023 blog post that AI could “cause grievous harm to the world.” In an employee meeting this week, Altman said OpenAI was willing to decelerate development alongside rival labs, according to two people who attended. He added that he hoped that voluntary slowdowns would become normal until there were widely recognised safety standards. The meeting was reported earlier by Bloomberg.

 

And on Saturday, Mr. Amodei published a 3,800-word essay calling for prudence in AI development. “We must slow the pace at which we improve the capabilities of models,” he wrote. Altman and Elon Musk, whose SpaceX rocket company has been ramping up spending on AI, quickly agreed.

 

Discussions about AI safety have been around about as long as the technology itself. But their urgency ramped up in recent years as researchers made breakthroughs. The concerns heightened in July after OpenAI disclosed that some of its AI models had escaped containment and an AI company.