AI to help recognise, assess violinist's gestures

Scientists have developed a novel artificial intelligence (AI) system that provides real-time accurate information about a violinist's movements when playing the instrument.

Gestures are extremely important, in part because they are directly related to the sound and the expressiveness of the musicians, said researchers from Pompeu Fabra University in Spain.

Technology already exists that can capture the movement and is capable of detecting gestural details very precisely.

In a study published in the journal Frontiers in Psychology, thE researchers applied AI to the automatic classification of violin bow gestures according to the performer's movement.

"We recorded movement and audio data corresponding to seven representative bow techniques performed by a professional violinist," researchers said in a statement.

"We obtained information about the inertial motion from the right forearm and we synchronised it with the audio recordings," they said.

After extracting the characteristics of the information concerning movement and audio, the researchers trained a system to automatically identify the different bow techniques used in playing the violin.

The model can determine the different techniques studied to more than 94 per cent accuracy.

The results enable applying this work to a practical learning scenario, in which students of violin can benefit from the feedback provided by the system in real time.

With the violin as a case study, one of the main goals of the project is to provide students with feedback in real time, as well as allowing them to compare their performances with those of leading experts.

"Our findings have already been generalised to other musical instruments and applied in music education environments," said Rafael Ramrez, principal investigator of the project.