Smart glove gives voice to gestures

Four engineering students from Bengaluru develop a glove that translates commonly used sign language in India into voice

Don't want to miss the best from Business Standard?

After college students from Ukraine, New Zealand, Malaysia and Washington, now four students from Amrita School of Engineering in Bengaluru have come up with a smart glove that can recognise commonly used gestures in India and translate these into voice.

The students call the smart glove Mudra.

Developed by Abhijith Bhaskaran, Anoop G Nair, Deepak Ram and Krishnan Ananthanarayanan, Mudra glove can be worn like any other riding glove. Once worn, the glove recognises hand gestures in all possible directions and angles, using flex resistors, accelerometer and gyroscope.

The corresponding output is transmitted as speech through inbuilt speakers. The glove can currently recognise numbers from one to 10, and gestures frequently used by Indians corresponding to words such as goodbye and thank you.

“We have used a combination of finger gestures and hand movement to help detect and understand an action. We have pre-fed 70 words that correspond to various actions in the glove. This can be extended further as well,” said Nair.

Initially, the team intended to use a camera device but as it proved to be bulky and expensive, they shifted to flex technology. Each flex sensor costs the team Rs 750. The team has used 10 flex sensors, an accelerometer and a gyroscope, taking the total cost to Rs 7,500. The camera device version would have cost the team at least Rs 70,000.

A prototype of the glove, built in 16 weeks, is being tested at the Amrita Robotics Research Lab.

The students say the most difficult part of the glove, also their final-year project, was the design.

“The design was crucial, as a stiff hold was required on the fingers. A range of values was calibrated precisely for each specific position of the finger and the rest was filtered out. This was a time consuming activity and had to be very accurate,” said Bhaskaran.

HOW THE SMART GLOVE WORKS

• The smart glove recognises commonly used gestures in India and translates it into voice
• Once worn, it recognises hand gestures in all possible directions and angles using flex resistors, accelerometer and gyroscope
• The corresponding output is transmitted as speech through inbuilt speakers
• It can recognise numbers from one to 10, and gestures frequently used by people corresponding to words such as goodbye, thank you

The movement of the hand posed another challenge. Although the inertial measurement unit offered values, these were not accurate, owing to noise. Filtering techniques were adopted for precision. Since differentiating between various orientations and movements of the hand with only one sensor was proving to be difficult, the students developed a novel method of state estimation.

The team is looking to raise funds, so that there can be mass production of the gloves and to make it commercially available. While the glove helps speech-and hearing-impaired people, it is multi-purpose as well. It can be reprogrammed for a range of applications in which motion-sensor technology plays an important role, such as gaming stations, virtual reality, remote control of devices, and the robotics and medical industry.

A similar glove using flex technology had also been developed by students in Ukraine. They used 15 flex sensors and gave the option to transfer the voice (recording) via Bluetooth to a mobile device of even a computer . The team, Team QuadSquad, also won the Microsoft Imagine Cup in 2012, according to reports in the Daily Mail.

The cost of the glove, which is called EnableTalk, is estimated to be $150 (around Rs 10,000).

A team of two students from the University of Washington also created a similar glove and called it SignAloud, according to the Daily Mail report. The cost of this device is not known. A collaborative group from New Zealand and Malaysia are also working on a sign-to-text program, which translates gestures from Malaysian sign language to multiple languages.

While the team from India says these might be more cost-effective, they have no complete research on what the other teams have done. “This is also one reason for not applying for patents,” Nair added.