First hidden screen-camera communication created

Researchers have created the first form of real-time communication that allows screens and cameras to talk to each other without the user knowing it.

Using off-the-shelf smart devices, the new system supports an unobtrusive, flexible and lightweight communication channel between screens (of TVs, laptops, tablets, smartphones and other electronic devices) and cameras.

The system called HiLight will enable new context-aware applications for smart devices, thus opening the way for new applications of such devices.

Such applications include smart glasses communicating with screens to realise augmented reality or acquire personalised information without affecting the content that users are currently viewing.

The system also provides far-reaching implications for new security and graphics applications.

In a world of ever-increasing smart devices, enabling screens and cameras to communicate has been attracting growing interest.

"The idea is simple: information is encoded into a visual frame shown on a screen, and any camera-equipped device can turn to the screen and immediately fetch the information," the authors from Dartmouth College in New Hampshire said.

The screen-camera communication is free of electromagnetic interference, offering a promising alternative for acquiring short-range information.

"Our work provides an additional way for devices to communicate with one another without sacrificing their original functionality," said senior author Xia Zhou, assistant professor and co-director of the DartNets (Dartmouth Networking and Ubiquitous Systems) Lab.

The findings are set to be presented at the ACM MobiSys 2015, a conference in mobile systems, applications and services, in Florence, Italy this week.