Indian AI tackles non-Eng language data scarcity with 'transfer learning'

Transfer learning enables a model trained on one task, like an English dataset, to be fine-tuned on new tasks or languages with less data, leveraging its pre-existing knowledge efficiently

With the success of ChatGPT and other generative artificial intelligence platforms, the need for building Indian language large language models (LLMs) is gaining speed. Though there have been several successes in creating Indian language LLMs, because of the variety of languages that the country exposes there is a need to look at other techniques. 

Industry players and AI practitioners believe that techniques like ‘transfer learning’ could be one way of solving the language conundrum. 

Transfer learning means a pre-trained model in some tasks or languages is reused as the starting point for a model on a second