Gmail's Gemini-powered summaries may expose users to security risks: Report

Reportedly, a researcher recently discovered a security flaw in Gmail's AI-generated summaries that could allow threat actors to display malicious links and messages

Google has been gradually integrating new AI capabilities into its mobile Gmail app. In June, it introduced a feature powered by Gemini that generates summaries of emails and lengthy threads. According to a report by The Indian Express, a recently uncovered security flaw indicates that these AI-generated summaries can be misused to display harmful instructions and embed links to malicious websites.

 

Indian Express cites Marco Figueroa, GenAI Bug Bounty Programs Manager at Mozilla, stating that a security researcher uncovered a prompt injection flaw in Google Gemini for Workspace, which let attackers “hide malicious instructions inside an email” that triggered when users clicked the “Summarise this email” button in Gmail.

 

Attack through Gemini: How does this work

As per the report, hackers found a way to hide secret instructions in emails that trick Google’s Gemini AI. They did this by placing hidden text at the end of the email using HTML and CSS, making the font size zero and the colour white so it could not be seen.

 

Because these emails do not contain attachments, they can easily pass through Google’s spam filters and reach users' inboxes. When someone opens the email and clicks “Summarise this email” using Gemini, the AI follows the hidden commands without knowing they are harmful.

 

These hidden instructions made Gmail display a fake phishing warning that appeared to come from Google. Since it looks like a real warning from Gmail itself, users are more likely to believe it.  

 

Mozilla’s Marco Figueroa explained how such prompt injections can be detected with:

• Gemini can be updated to ignore or remove hidden text in emails.
• Google can use a post-processing filter to scan Gemini’s output for: Urgent messages, Phone numbers, Suspicious links.

These flagged elements can then be reviewed for potential threats.

 

Google has reportedly issued a statement to BleepingComputer, stating, “We are constantly hardening our already robust defenses through red-teaming exercises that train our models to defend against these types of adversarial attack."

 

The company representative clarified to BleepingComputer that some of the mitigations are in the process of being implemented or are about to be deployed.

 

The report further states that Google has seen no evidence of incidents manipulating Gemini in the way demonstrated in Figueroa's report. 

Hackers often try to stay ahead. So, it is ideal not to completely rely on AI-generated summaries. Always double-check links and email content before clicking.