New app allows classical musicians to perform solo with virtual orchestra

A new computer system allows musicians to perform as solo with a full philharmonic orchestra from the comfort of their own living rooms.

Christopher Raphael, chair of Computer Science in the School of Informatics and Computing at Indiana University, Bloomington, and a former professional oboist, said classical musicians spend untold hours learning to play the solo literature featuring their instrument, but very few ever perform this music with the accompanying ensemble.

He said that the reason is it takes many players to make an orchestra, but only one to be a soloist, explaining while the oboe is not the favorite solo instrument of composers or audiences, he has performed as a soloist 10 or 15 times.

Raphael said that he found the experience was thrilling, and so he wanted to find a way to replicate the feeling of this experience, and to share it with others.

To model the hearing of the accompanists-and thus be able to identify, and respond to, the notes played by the soloist, and when they occur-the system uses an algorithm known as a hidden Markov model, which is commonly employed in speech-recognition technologies.

The simulated orchestra is synthesized from a prerecorded orchestra, which means there is no limit to the number of instruments involved, "though it isn't always easy to find a recording of a concerto minus the soloist," Raphael says.

The system-which Raphael has dubbed the "Informatics Philharmonic"-is designed to understand the "imprecise nature" of humans, and, like an artificial intelligence system, can learn to adapt to the soloist's interpretation of the music.

The model can be automatically trained from past performances (and must be trained for each individual soloist), "thus capturing the essence of the human rehearsal process in which one learns from example," he said.