Soon: computers with common sense

Researchers, including those of Indian-origin, have developed a new programme that helps computers to learn common sense through pictures, enabling them to scour the internet and label images on its own.
The computer programme can make associations between things to obtain common sense information that people just seem to know without ever saying - that cars often are found on roads, that buildings tend to be vertical and that ducks look sort of like geese.
The computer programme called the Never Ending Image Learner (NEIL) is running 24 hours a day at Carnegie Mellon University, searching the Web for images, doing its best to understand them on its own and, as it builds a growing visual database, gathering common sense on a massive scale.

NEIL leverages recent advances in computer vision that enable computer programmes to identify and label objects in images, to characterise scenes and to recognise attributes, such as colours, lighting and materials, all with a minimum of human supervision.
In turn, the data it generates will further enhance the ability of computers to understand the visual world.
Based on text references, it might seem that the colour associated with sheep is black, but people - and NEIL - nevertheless know that sheep typically are white.

"Images are the best way to learn visual properties," said Abhinav Gupta, assistant research professor in Carnegie Mellon's Robotics Institute.

"Images also include a lot of common sense information about the world. People learn this by themselves and, with NEIL, we hope that computers will do so as well," said Gupta.
A computer cluster has been running the NEIL programme since late July and already has analysed three million images, identifying 1,500 types of objects in half a million images and 1,200 types of scenes in hundreds of thousands of images.
It has connected the dots to learn 2,500 associations from thousands of instances.
Abhinav Shrivastava, a PhD student in robotics, said NEIL can sometimes make erroneous assumptions that compound mistakes, so people need to be part of the process.

A Google Image search, for instance, might convince NEIL that "pink" is just the name of a singer, rather than a colour.
"People don't always know how or what to teach computers. But humans are good at telling computers when they are wrong," Shrivastava said.
People also tell NEIL what categories of objects, scenes, etc, to search and analyse. But sometimes, what NEIL finds can surprise even the researchers.
It can be anticipated, for instance, that a search for "apple" might return images of fruit as well as laptop computers.
But Gupta and his team had no idea that a search for F-18 would identify not only images of a fighter jet, but also of F18-class catamarans.