OpenAI staff warned of 'dangerous' AI tool: Sam Altman was fired next day

By Anna Tong, Jeffrey Dastin & Krystal Hu

A day after Sam Altman sacked the OpenAI board that fired him, it has been revealed that several staff researchers of the firm wrote a letter to the board of directors warning of a powerful artificial intelligence (AI) discovery that they said could threaten humanity, two people familiar with the matter told Reuters.
 

The previously unreported letter and AI algorithm were key developments before the board’s ouster of Altman, the poster child of generative AI, the two sources said. The sources cited the letter as one factor among a longer list of grievances by the board leading to Altman’s firing, among which were concerns over commercialising advances before understanding the consequences.

Reuters was unable to review a copy of the letter. After being contacted by Reuters, OpenAI, which declined to comment, acknowledged in an internal message to staffers a project called Q* and a letter to the board before the weekend's events, one of the people said. 

 

Some at OpenAI believe Q* (pronounced Q-Star) could be a breakthrough in the startup’s search for what's known as artificial general intelligence (AGI), one of the people told Reuters. OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks. Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because the individual was not authorised to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. Reuters could not independently verify the capabilities of Q* claimed by the researchers. 
 

In their letter to the board, resea­rchers flagged AI’s prowess and potential danger, the sources said without specifying the exact safety concerns noted in the letter. 
 

Altman led efforts to make ChatGPT one of the fastest growing software applications in history and drew investment — and computing resources — necessary from Microsoft to get closer to AGI.
 

“Four times now in the history of OpenAI, the most recent time was just in the last couple weeks, I’ve gotten to be in the room, when we sort of push the veil of ignorance back and the frontier of discovery forward, and getting to do that is the professional honour of a lifetime,” he said at the Asia-Pacific Economic Cooperation summit.
 

A day later, the board fired Altman.