Homegrown e-commerce giant Flipkart has partnered with the Indian Institute of Science (IISc), the country’s premier research institute, to build a Google-like ‘knowledge graph’. This will help the company to more efficiently catalogue its 380 million and growing listings, aiding customers to find the right products which will boost its business.
Headed by Partha Talukdar, Assistant Professor in the Department of Computational and Data Sciences (CDS) at IISc, the project is one of Flipkart’s grand vision plans for which it is partnering with academia. The company has several such other ongoing collaborations with premier Indian institutes as well as foreign universities to develop machine learning (ML), voice recognition and even delivery using drones.
By utilising Natural Language Processing (NLP), a field in which Talukdar has around 15 years of experience, Flipkart wants to analyse product descriptions given by sellers and turn it into a rich descriptions that customers demand. Going forward, the solution that’s being jointly developed, will be able to categorise products, find relations between them and ultimately could power smarter recommendations on Flipkart.
“We get a lot of catalogue data from sellers on Flipkart and one of the problems they face is that they have to adhere to our standards of describing something. We want to make that process simpler for sellers and want to ask them for the least amount of information, but at the same time provide our customers with as much data about a product as possible to make a decision,” says Nishant Khurana, Senior Engineering Manager at Flipkart, who is working on the new project.
It’s this gap between asking for little but delivering a lot that Flipkart is trying to address at first and wants to use NLP techniques developed by Professor Talukdar and his team to analyse text written by a human to extract relevant information about a product from it. This data can then be used to link two or more products to each other no matter how disparate they may seem using other ML and AI techniques.
Essentially Flipkart is trying to use these smarter ways of scraping data to build a knowledge graph, a term for the underlying base technologies that was made popular by search engine giant Google to enhance results it shows users from a variety of sources.
“Our research has been focused on how we can read unstructured data at scale which is written in natural language. We have done quite a bit of work on reasoning and canonicalization which is applicable to this space. The way to understand it is that Flipkart has some data and consumers are trying to access this to discover products, so you need to have one common language to put everyone on the same page,” says Talukdar.
Collaborating with academia, Flipkart gets access to years of research and some of the leading minds in the field to build such technologies. For researchers, the access to vast amounts of data which are up-to-date is among the biggest lures. Talukdar says such projects allow him and his team to get out of the “ivory towers” and work on solving real-world problems.
Several of the world’s largest technology companies including Google, Facebook and Microsoft also work with top-notch universities to develop intellectual properties. The credits for such research is generally shared between the companies and the researchers.
Mayur Datar, who heads Flipkart’s data sciences division, has a team of several such researchers building future solutions that will drive the company’s growth. However, Flipkart continues to look outward for innovation as there are far too many problems to solve and there are certain areas of research where internal teams might be lacking in expertise.
However, working with internal teams or external researchers, Datar says he’s clear that the future of Flipkart is going to be determined by how well it adopts ML and AI. As the Vice President of Engineering, he says he’s pushing more young engineers to dip their feet in newer technologies and to take up courses in AI and ML.
“We have obtained certain amount of scale and profitability using human intelligence and domain knowledge that people bring. But we can continue down this road for maybe another year or two, but beyond that if we do not embrace AI and ML, it’s absolutely clear that we will not be able to sustain as a company,” said Datar.