Modi-Fridman podcast: How AI voice cloning helped break language barriers

Popular podcaster and AI researcher Lex Fridman hosted PM Modi for an interview, featuring AI-powered Hindi and English dubbing by voice cloning startup ElevenLabs

"Artificial intelligence (AI) can create many things based on human imagination," said Prime Minister Narendra Modi on computer scientist and AI researcher Lex Fridman's podcast released on Sunday, that used AI-powered voice cloning technology for enhancing accessibility and reach of the conversation.

 

The podcast's dubbing was done by a US-based startup ElevenLabs. The software company used AI voice cloning technology that kept PM Modi's and Fridman's voice authentic even in translations, including intonations, emotions, and style of speech.

 

Currently, the audio and subtitles of the podcast are available in Hindi, English, and Russian (and soon other languages). It is also available in the original (mix of Hindi and English).  Read: Soon, Gemini will replace Google Assistant on Android phone: What to expect

 

Mati Staniszewski, the co-founder of ElevenLabs that specialises in developing natural-sounding speech synthesis software using deep learning, said in a post on X, "Inspired by Lex Fridman and Narendra Modi's conversation - so proud of our team for making it accessible in both English and Hindi. ElevenLabs was founded on the back of poor-quality dubbing. Seeing it evolve into an immersive experience that preserves emotion and intent is incredible."

 

 

The three-hour conversation focused on PM Modi's journey from childhood to becoming the Prime Minister of India and touched upon a wide array of topics including leadership, diplomacy, democracy in India, poverty, US President Donald Trump, China, and 2002 Gujarat riots.

 

Talking about AI during the podcast, PM Modi said that without genuine human intelligence, AI cannot thrive or progress sustainably. He highlighted that AI is fundamentally about collaboration, and India is making a significant contribution to its development. "No matter what the world does with AI, it will remain incomplete without India. I'm making this statement very responsibly," he said.

 

What is AI voice cloning?

 

According to ElevenLabs, AI dubbing is the cutting-edge practice of using generative AI, to seamlessly translate spoken content into different languages. Voice cloning replicates the unique characteristics of the original speaker's voice, ensuring authenticity in dubbed content.