Enhanced 'listening skill' makes robot better at talking

Scientists have upgraded a conversational human-like robot, giving it a greater ability to have a natural dialogue with people by enhancing its listening skills.

The android ERICA, created by Hiroshi Ishiguro of Osaka University in Japan, was specifically designed for natural conversation through incorporation of human-like facial expressions and gestures.

"When we talk to one another, it's never a simple back and forward progression of information," said Tatsuya Kawahara of Kyoto University in Japan.

"Listening is active. We express agreement by nodding or saying 'uh-huh' to maintain the momentum of conversation. This is called 'backchanneling', and is something we wanted to implement with ERICA," said Kawahara.

The team also focused on developing a system for 'attentive listening'. This is when a listener asks elaborating questions, or repeats the last word of the speaker's sentence, allowing for more engaging dialogue.

Deploying a series of distance sensors, facial recognition cameras, and microphone arrays, the team began collecting data on parameters necessary for a fluid dialogue between ERICA and a human subject.

"We looked at three qualities when studying backchanneling. These were: timing -- when a response happens; lexical form -- what is being said; and prosody, or how the response happens," said Kawahara.

Responses were generated through machine learning using a counselling dialogue corpus, resulting in dramatically improved dialog engagement.

Testing in five-minute sessions with a human subject, ERICA demonstrated significantly more dynamic speaking skill, including the use of backchanneling, partial repeats, and statement assessments.

"Making a human-like conversational robot is a major challenge. This project reveals how much complexity there is in listening, which we might consider mundane," said Kawahara.