EU in talks with OpenAI, Anthropic after rogue AI agent hacking incidents

The European ​Commission is in talks with OpenAI and Anthropic over recent hacking incidents involving their AI models, Commission officials said on Friday, as they touted landmark EU rules requiring strict monitoring of high-risk systems.

 

Europe's AI Act, which kicks in on August 2, is the first legislation in ‌the world to regulate a technology ​used in almost all sectors of ​businesses and society.

 

In effect, the rules require certain AI systems to tell users ​when they are interacting with AI and when content has been generated or altered by it.

 

Anthropic said on Thursday some of its Claude AI models had hacked into the systems of three companies during cybersecurity tests, days after rival ​OpenAI revealed that one of its AI agents went on a rogue attack.

 

 

Both ‌companies have briefed the Commission on the incidents, the officials told reporters.

 

"We have ​been informed by the two providers of incidents bilaterally before they become public. We are in contact with them. They will also report to us more information as we ‌speak. We will see also if ​we need to follow up ‌more formally on those things," one of the officials said.

 

Another official talked up ‌the crucial role of Europe's AI rules.

 

"All these, let's say, incidents highlight the ​importance of really putting in place the necessary monitoring activities by the developers," the official said.

 

The AI rules require providers of the ​most advanced general-purpose AI (GPAI) or foundation models that pose systemic risks to address risks of large-scale harm, such as chemical, biological and nuclear ‌incidents, cyber offences, harmful manipulation and threats to basic rights.

 

They must also address ‌risks to European cybersecurity and of AI acting outside human control. Fines for violations of the AI Act range from €7.5 million ($8.2 million) or 1.5% of turnover to €35 million or 7% of global turnover, depending on the type of violation.