Robot avatars that use gestures can be better understood

By programming robot avatars to "talk with their hands," we can understand them as well as we do our fellow human beings, a new study has found.
Avatars have been in existence since the 1980s and today are used by millions of people across the globe, researchers said.
From artificial intelligence to social media and psychotherapy to high-end video games, they are used to sell things, to solve problems, to teach us and to entertain us, they said.
As avatars become more sophisticated, and their use in society grows, research is focusing on how to improve communication with them.
Getting your message across with your avatar is more important than ever, and learning how to improve this communication is a big deal, researchers said.

Paul Bremner from University of The West of England and Ute Leonards from University of Bristol in the UK built their study around the hypothesis that if avatars were to use "iconic" hand gestures together with speech, we would understand them more easily.

Iconic gestures have a distinct meaning, like opening a door or a book, and using gestures together with speech is known as "multi-modal communication."
The aim of the study was to discover if people could understand avatars performing multi-modal communication as well as they could a human actor.
The study also investigated if multi-modal communication by an avatar was more understandable than speech alone.

To test their theory, the scientists filmed an actor reading out a series of phrases while performing specific iconic gestures.
They then filmed an avatar using these recorded phrases and mimicking the gestures. Films of both the actor and avatar were then shown to the experiment participants, who had to identify what the human and avatar were trying to communicate.
The scientists were able to prove that multi-modal communication by avatars is indeed more understandable than speech alone. Not only that, but when using multi-modal communication, we understand them as well as we do humans.
While performing the gestures, the actor used state-of-the-art technology. His movements were tracked using a Microsoft Kinect sensor so that his arm gestures could be recorded as data.

The avatar used this data to mimic his gestures. Despite a few limitations, the research showed that the method of translating human gestures to an avatar was successful.
Researchers are confident that the avatar gestures, when used with speech, are as easily understood as from a human.
The study was published in the journal Frontiers in Psychology.