Yamaha AI Turns Dancer's Movement Into Piano Sounds

Yamaha has created an AI that can do something very strange. The AI is integrated into a new musical system that Yamaha created that is designed specifically to turn the motions of a world-renown dancer into music that comes from a real piano. Dancer Kaiji Moriyama was able to control the piano with his movements.

The concert that Moriyama performed at was called "Mai Hi Ten Yu" and was sponsored by the Tokyo University of the Arts and Tokyo University of the Arts COI. Yamaha provided the original music system for the performance able to turn human motion into music using an AI and sensors connected to the dancer's body.

The dancer and his piano music were accompanied by players from the Berlin Philharmonic Orchestra Scharoun Ensemble. Yamaha sees its new musical system as progress in the pursuit of new forms of artistic expression and plans to develop the system further.

The AI in the system is still under development and is able to identify the movement of the dancer in real time by analyzing signals from four sensor types attached to the body. The database links melody and movements and the AI is able to create melody data (MIDI) from movements instantly.

That MIDI data was then sent out to a Yamaha Disklavier player piano and translated into music. Yamaha says that piano is able to produce rich sounds with very slight changes in piano touch.

SOURCE: Yamaha