This 'vegebot' uses machine learning to harvest lettuce

Cambridge scientists have developed a vegetable-picking robot that uses machine learning to identify and harvest a commonplace but challenging agricultural crop.

Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.

The 'Vegebot', developed by a team at the University of Cambridge in the UK, was initially trained to recognise and harvest iceberg lettuce in a lab setting.

It has now been successfully tested in a variety of field conditions in cooperation a local fruit and vegetable co-operative, according to the study published in The Journal of Field Robotics.

Although the prototype is nowhere near as fast or efficient as a human worker, it demonstrates how the use of robotics in agriculture may be expanded, even for crops like iceberg lettuce which are particularly challenging to harvest mechanically.

Crops such as potatoes and wheat have been harvested mechanically at scale for decades, but many other crops have to date resisted automation. Iceberg lettuce is one such crop.

Although it is the most common type of lettuce grown in the UK, iceberg is easily damaged and grows relatively flat to the ground, presenting a challenge for robotic harvesters.

"Every field is different, every lettuce is different," said Simon Birrell from Cambridge's Department of Engineering.

"But if we can make a robotic harvester work with iceberg lettuce, we could also make it work with many other crops," Birrell said.

"At the moment, harvesting is the only part of the lettuce life cycle that is done manually, and it's very physically demanding," said Julia Cai, who worked on the Vegebot while she was an undergraduate student in the lab of Fumiya Iida at Cambridge.

The Vegebot first identifies the 'target' crop within its field of vision, then determines whether a particular lettuce is healthy and ready to be harvested.

It finally cuts the lettuce from the rest of the plant without crushing it so that it is 'supermarket ready'.

"For a human, the entire process takes a couple of seconds, but it's a really challenging problem for a robot," said Josie Hughes, co-author of the study.

The Vegebot has two main components: a computer vision system and a cutting system.

The overhead camera on the Vegebot takes an image of the lettuce field and first identifies all the lettuces in the image, and then for each lettuce, classifies whether it should be harvested or not.

A lettuce might be rejected because it is not yet mature, or it might have a disease that could spread to other lettuces in the harvest, researchers said.

They developed and trained a machine learning algorithm on example images of lettuces.

Once the Vegebot could recognise healthy lettuces in the lab, it was then trained in the field, in a variety of weather conditions, on thousands of real lettuces.

A second camera on the Vegebot is positioned near the cutting blade, and helps ensure a smooth cut.

The researchers were also able to adjust the pressure in the robot's gripping arm so that it held the lettuce firmly enough not to drop it, but not so firm as to crush it.

The force of the grip can be adjusted for other crops.

"We wanted to develop approaches that weren't necessarily specific to iceberg lettuce, so that they can be used for other types of above-ground crops," said Iida, who leads the team behind the research.

In future, robotic harvesters could help address problems with labour shortages in agriculture, and could also help reduce food waste, researchers said.