Picking apples and oranges: New quality check comes to grocery shops

Amazon Fresh says its application to check freshness of fruits, vegetables on Indian shelves has opportunities abroad

In Sainikpuri, a neighbourhood in Northeast Hyderabad, Amazon Fresh worker Rohit Kumar is picking up fruits and vegetables at a grocery store. He takes out his phone and scans the produce with an app called Johari to identify defects like cuts and cracks.

“Johari app helps us a lot in minimising the time taken for sorting and grading as it does half the work for us by identifying defects,” said Kumar about his work for the fresh grocery delivery service. 
 

Johari is part of a store shelf-monitoring solution (a machine learning-powered quality assurance system for fresh produce) that Amazon India unveiled recently. With the app, the e-commerce company aims to enable its sellers to meet Amazon Fresh consumers' demand for quality fresh fruits and vegetables. The shelf-monitoring solution is a "game changer" in monitoring and maintaining quality, said the firm.

“Our team is based in India. We were trying to explore a new application for computer vision. The business required technology of this kind because they already had manual assessments on the ground and automation was attractive to pursue,” said Rajeev Rastogi, vice president, machine learning, Amazon, in an interview. “Our first launch of the (Johari app) is in India, but clearly this technology applies to every country in the world. We are exploring opportunities in other marketplaces as well.”

Identifying defects in fresh produce requires time and expertise. Technology makes it easier for Amazon Fresh workers to use their judgment in that task. Johari automates and standardises the defect-identification part of sorting and grading fresh produce. It is expected to reduce work time by about 50 per cent. 

“For our grocery business, (quality) is the single most important customer input and also the number one driver of repeat purchases,” said Rastogi. “Manually examining each produce like every potato, tomato and onion is not really scalable to millions of quality assessments per day. It is error-prone, costly and causes fatigue in humans. Our solution has the ability to carry out millions of assessments per day and at a cost far below that of any other method.”

The monitoring solution is powered by computer vision models and Wi-Fi-enabled IoT (Internet of Things) cameras to detect pre-determined defects in fruits and vegetables. It identifies specific visual defects such as cuts, cracks and pressure damage. 

Amazon has developed two models: One for detecting each item in a crate and counting the number of items in it, and the other for identifying defects in each item. Both models are trained using annotated defects in millions of images.

The solution supports manual monitoring through the Johari app and an automated version using cameras installed on top of produce shelves. In manual monitoring, operators use the app on their smartphone to submit images of a crate containing produce. The solution assesses the image for quality and then to detect defects. It uses a grading logic to highlight items that don’t meet quality criteria. Using information they get back, workers cull a rejected produce and then update the app about their action. It takes some six seconds to assess a crate of produce.

In automated monitoring, cameras installed in store shelves take images automatically. 

The monitoring solution is crucial for Amazon as grocery is one of its fastest-growing businesses. Amazon Fresh, available in more than 60 cities, delivers wet and dry grocery products to consumers. 

“Anything that can help to ensure that your quality is very high is a game changer,” said Rastogi.