Now, draw sketches to search for images, videos!

Scientists have developed a new system that uses hand-drawn sketches on a tablet or interactive paper to search for matching image or video in the database.
People today are increasingly confronted with the challenge of having to find their way around vast collections of photos and videos, both in their work lives and at home.
Although search engines such as Google and Bing make it easy to find documents or websites quickly and efficiently using search terms, the options for searching collections of multimedia objects are more limited.
Researchers at the University of Basel in Switzerland have developed a system known as 'vitrivr', which allows a search for images and videos by means of a sketch.

The user creates a sketch of the desired object on a tablet or interactive paper, and the program delivers the images and video clips that most resemble it.
For videos, the user can even specify on the sketch in which direction an object is moving in the searched sequence.

In designing the system, the researchers set a very broad similarity concept and adapted it to different types of sketch; for example, similar colours, shapes or directions of movement.
Individual searches can then be augmented by a range of other query types - search terms, examples of images and videos, or combinations of all these.

An important feature of the new system is its scalability, a feature that means it can be used even with very large multimedia collections.
The vitrivr system is open source and freely available to the international research community. It may be used for a wide range of purposes, from discerning patterns of movement in sports videos to searching collections of digital watermarks.