Building robust search engine for users at top of the Flipkart's agenda

Home-grown e-commerce major Flipkart is consolidating its efforts in data science and engineering by focusing on few key areas such as personalisation and product discovery through search. The move is aimed at building for long-term competitiveness. 

Terabytes of data is coughed out every day by Flipkart on millions of users who visit or transect on the e-commerce site. At the back-end, this data is intricately studied to derive buying trends, under behaviour and preferences, and popular products and categories. The data also helps Flipkart understand the users better allowing it to serve product recommendations, ads, and offers uniquely to each user.

“We have a big appetite for data sciences, we doubled ourselves in the last year, and have very aggressive plans for this unit going forward,” Flipkart’s chief data scientist Mayur Datar told Business Standard. The firm now has 45 data scientists that work in tandem with engineering teams across functions.

Instead of functioning as one department, the company is embedding the data scientists into various engineering units who are report in to the engineering leaders of those respective units. “The kind of operation structure that I have developed is that the leader, the senior-most person in each of this functional units, reports to me,” Datar said.

Of the many projects bring driven at Flipkart, Datar said, the product search in an area where good amount resources are being deployed. “Search is one of our most important features. You talk to us through that search box,” and the team is working on language models that understand search queries in colloquial lingos, and deliver accurate results.

“When you look at tier II and tier III cities, Flipkart is very strong in these markets. You and I have grown up in metros and know what a ‘cart’ is or what ‘check-out’ is’; we are basically English educated. These are people who probably don’t speak English, are looking at an English app, and it has a search button there. They sometimes type things like ‘kala juuta, ‘shaddi lehenga’ or ‘sleeveless blouse’ while searching for goods. They might not even know the right spelling of ‘jeans’. So you have to make sense of those queries,” explained Datar.

The models being worked on may ultimately be able to understand queries running the length of a sentence. Flipkart is also preparing to launch voice search capabilities. “Last year, we acquired a company called Liv.ai. We are making huge investments into that. In a year or so, you’ll see a lot of announcements (related to search),” Datar said.

The other big effort is personalised recommendations. While personalised app screens is an old science, measuring the success of personalised recommendations and the process of fine-tuning the recommendations is a work in progress. 

"If I show a content to a particular user, how likely is the user to engage with that content. It's traditionally measured in terms of click rate. But to go beyond click-rate and beyond click, will you actually go ahead and buy that product? Estimating these quantities is very critical to optimising many aspects, recommendations, optimising search, optimising search, optimising advertising," said Datar.

Data science is also central to some of the core functions like forecasting demand, inventory management, and warehouse management. These operations are being optimised on an on-going basis, said Datar.

Of about 400 million internet users in India, only 90 million shop online, according to estimates from RedSeer, an e-commerce focussed consultancy. With new demand envisaged from smaller towns and cities, a convenient and useful shopping experience may decide the shopping brand stick with.