The Brain Opera Technology: New Instruments and Gestural Sensors for Musical Interaction and Performance

This paper describes the array of new musical instruments and interactive installations developed for the Brain Opera, a large, touring multimedia production, where the audience first explores a set of musical modes at a variety of novel, interactive stations before experiencing them in an actual performance. Most of the Brain Opera's installations were intended for the general public, employing different gestural measurements and mappings that allow an untrained audience to intuitively interact with music and graphics at various levels of complexity. Another set of instruments was designed for a trio of trained musicians, who used more deliberate technique to perform the composed music. This paper outlines the hardware and sensor systems behind these devices: the electric field sensors of the Gesture Wall and Sensor Chair, the smart piezoelectric touchpads of the Rhythm Tree, the instrumented springs in Harmonic Driving, the pressure-sensitive touch screens of the Melody Easels, and the multimodal Digital Baton, containing a tactile interface, inertial sensors, and precise optical tracker. Also discussed are a set of controllers developed for the Brain Opera, but not currently touring with the production, including the Magic Carpet (immersive body sensing with a smart floor and Doppler radar) and an 8-channel MIDI-controlled sonar rangefinder.