Make a virtual musical instrument out of post-it notes

AR Music Kit lets anyone create and play a virtual musical instrument using their smartphone camera, paper notes and any object.

Musical pioneers have often embraced tech as a means of innovating their sound — synths are just one example. In that tradition, inventor Yuri Suzuki has created AR Music Kit — an incredibly accessible system that uses the smartphone to enable anyone to create a virtual musical instrument out of any object.

To begin, players print out chord codes or notes onto squares of paper and attach them to any object. They could make a guitar out of broomstick or a piano out of their desk. Next, they launch their smartphone camera through the app and select the sound they want to make. Then, they stand in front of their camera and ‘play’ the instrument by physically covering the chord they want to play. The app reads the sequence and translates it into music in real time. The AR Music Kit can also be used to make a music box by arranging the codes on a spinning surface — a record player for example — and capturing the pattern as it spins to create an ongoing melody.

The AR Music Kit was created as part of Android Experiments I/O challenge. How else could physical and digital tools be combined to encourage musical creativity?