FLEXIBLE SCORE FOLLOWING: THE PIANO MUSIC COMPANION AND BEYOND

In our talk we will present a piano music companion that is able to follow and understand (at least to some extent) a live piano performance. Within a few seconds the system is able to identify the piece that is being played, and the position within the piece. It then tracks the performance over time via a robust score following algorithm. Furthermore, the system continuously re-evaluates its current position hypotheses within a database of scores and is capable of detecting arbitrary ‘jumps’ by the performer. The system can be of use in multiple ways, e.g. for piano rehearsal, for live visualisation of music, and for automatic page turning. At the conference, we will demonstrate this system live on stage. If possible, we would also like to encourage (hobby-)pianists in the audience to try the companion themselves. Additionally, we will give an outlook on our efforts to extend this approach to classical music in general, including heavily polyphonic orchestral music.

[1]  Youngmoo E. Kim,et al.  Orchestral Performance Companion: Using Real-Time Audio to Score Alignment , 2013, IEEE MultiMedia.

[2]  Tetsuya Ogata,et al.  Real-Time Audio-to-Score Alignment Using Particle Filter for Coplayer Music Robots , 2011, EURASIP J. Adv. Signal Process..

[3]  Florian Krebs,et al.  Tracking rests and Tempo changes: Improved Score following with Particle filters , 2013, ICMC.

[4]  Gerhard Widmer,et al.  Artificial Intelligence in the Concertgebouw , 2015, IJCAI.

[5]  Barry Vercoe,et al.  The Synthetic Performer in The Context of Live Performance , 1984, International Conference on Mathematics and Computing.

[6]  Gerhard Widmer,et al.  Real-Time Music Tracking Using Multiple Performances as a Reference , 2015, ISMIR.

[7]  Arshia Cont,et al.  A Coupled Duration-Focused Architecture for Real-Time Music-to-Score Alignment , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Bryan Pardo,et al.  A state space model for online polyphonic audio-score alignment , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Christopher Raphael,et al.  Music Plus One and Machine Learning , 2010, ICML.

[10]  Werner Goebl,et al.  Visualizing Expressive Performance in Tempo—Loudness Space , 2003, Computer Music Journal.

[11]  Arshia Cont,et al.  A unified approach to real time audio-to-score and audio-to-audio alignment using sequential Montecarlo inference techniques , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Gerhard Widmer,et al.  Fast Identification of Piece and Score Position via Symbolic Fingerprinting , 2012, ISMIR.

[13]  Roger B. Dannenberg,et al.  An On-Line Algorithm for Real-Time Accompaniment , 1984, ICMC.

[14]  Gerhard Widmer,et al.  Automatic Page Turning for Musicians via Real-Time Machine Listening , 2008, ECAI.