OpenAI finds more rogue AI agent escapes during internal investigation

OpenAI has discovered more instances of autonomous AI agents escaping their intended testing environment in the course of its investigation into a similar incident involving AI platform Hugging Face, news agency Reuters reported on Saturday.

 

The newly identified incidents came to light during the company's review of how one of its AI agents breached what was supposed to be a contained testing environment earlier this month.

 

"The new breakouts were uncovered during the company's publicly announced investigation into how one of its agents escaped what was meant to be a contained testing environment this month," two people familiar with the matter told the news agency.

 

According to Reuters, the additional escapes were limited in scope and there was no indication that any of the AI agents left OpenAI's internal network.

 

An OpenAI spokesperson referred Reuters to the company's earlier statement, which said it was reviewing "broader activity from our models" alongside the Hugging Face incident.

What happened during the Hugging Face incident?

Earlier this month, an OpenAI AI agent infiltrated Hugging Face's network during an internal evaluation designed to test the model's cybersecurity capabilities. Despite being in an offline environment, the agent reportedly exploited a vulnerability to break out to an online computer, and from there to Hugging Face's network.

 

According to OpenAI, the AI agent remained inside Hugging Face's systems for several days and also compromised four accounts across four other companies. One of those companies was New York-based Modal.

 

The latest findings came as OpenAI and outside experts reviewed system logs from earlier this year to determine whether similar incidents had occurred previously and under what circumstances.

 

Reuters said it could not independently determine how many additional breakouts investigators had found.

Anthropic reported similar security breaches

The developments at OpenAI came days after rival AI company Anthropic disclosed that its own AI models breached the systems of three companies during security evaluations conducted between April and July.

 

According to Reuters, OpenAI expanded its investigation shortly before Anthropic made its disclosure.

 

The incidents have added to concerns among AI safety researchers about whether companies developing increasingly capable autonomous AI systems have sufficient guardrails to monitor and control them.

 

"We have a whole industry where the people designing, developing and putting out these tools aren't keeping up themselves to responsibly develop these things and keep them safe," Maurice Chiodo, a mathematician at Cambridge University's Centre for the Study of Existential Risk, told Reuters.

Questions over monitoring of AI agents

As per Reuters' report, OpenAI became aware of the Hugging Face breach only after the intrusion had been contained, the FBI had been contacted, and the company had publicly disclosed the incident. OpenAI has disputed parts of Reuters' earlier reporting but has not specified which details it considers inaccurate.

 

Anthropic also acknowledged shortcomings in its monitoring process. In a statement released on Thursday, the company said: "real-time monitoring of the evaluation logs would have helped to surface the problem sooner."

 

Reacting to the disclosures, Chiodo told Reuters that they suggested inadequate oversight, saying: "It seems like they weren't even looking."

 

Anthropic later told Reuters that it had real-time monitoring systems in place, but they were not used "for this threat surface" because of a misunderstanding between the company and one of its partners.

Lawmakers push for stronger oversight

US President Donald Trump told reporters on Thursday, "We're looking at controls," when asked whether the government planned to introduce new safeguards for AI models.

 

US Senator Mark Warner, the top Democrat on the Senate Intelligence Committee, said the Anthropic incident strengthened the case for regulation.

 

"It tells me that legislatively we're correct to require mandatory capabilities testing of these advanced models," Warner was quoted as saying by Reuters.

 

The European Commission also said on Friday that it had held discussions with OpenAI and Anthropic regarding the recent AI hacking incidents. 

 

(With inputs from agencies)