Google adds multimodal search to AI mode, rolls out image-based search

Google said this new enhancement will let users upload an image, ask questions about it and receive detailed, context-rich answers, complete with helpful links for further exploration

Google said it is expanding access to its AI Mode feature, previously exclusive to Google One AI Premium subscribers, to a wider group of Labs users in the US, following strong early feedback. With its broader rollout now underway, Google is also introducing a major update: multimodal search capabilities powered by Google Lens and a custom version of Gemini, its AI model. This enhancement allows users to take or upload an image, ask a question about it, and receive detailed, context-rich answers, complete with helpful links for further exploration.

 

The new feature goes beyond basic visual search. AI Mode now analyses not just the image but the overall context, which essentially covers understanding object relationships, textures, colours, shapes, and layout. Using Lens technology, it identifies specific items in the image, and with a technique known as “query fan-out,” the system runs multiple searches to gather deeper insights.

 

 

Google says the result is a more nuanced and relevant response than traditional search, allowing users to better understand what they’re seeing and make informed decisions. 

ALSO READ | Google Gemini Live update brings screen, camera awareness to these phones

  Google shared an example to showcase this feature. In the example shared, AI Mode interpreted the details within the image, accurately recognising each book on the shelf. It then generated relevant queries to gather information about those titles and find similar, highly rated suggestions. The response included a curated list of book recommendations, complete with links for additional details and purchasing options. Users will be free to ask follow-up questions to refine their search.

 

Google rolled out AI mode for select users on March 25. The US-based software company, in its blog post said that following the rollout, users have praised AI Mode for its clean interface, fast response times, and ability to handle complex queries.

 

According to Google’s blog post, questions asked in AI Mode are generally twice as long as those entered in standard Google Search, with many users relying on it for open-ended tasks like product comparisons, how-to guides, and travel planning.