Multimodal Sensor Analysis of Sitar Performance: Where is the Beat?

In this paper we describe a system for detecting the tempo of sitar performance using a multimodal signal processing approach. Real-time measurements are obtained from sensors on the instrument and by wearable sensors on the performer's body. Experiments comparing audio-based and sensor-based tempo tracking are described. The real-time tempo tracking method is based on extracting onsets and applying Kalman filtering. We show how late fusion of the audio and sensor tempo estimates can improve tracking. The obtained results are used to inform design parameters for a real-time system for human-robot musical performance.

[1]  George E. Lewis.  Too Many Notes: Computers, Complexity and Culture in Voyager , 2000, Leonardo Music Journal.

[2]  Ali Momeni,et al.  OpenSound Control: State of the Art 2003 , 2003, NIME.

[3]  Ajay Kapur,et al.  The Electronic Sitar Controller , 2004, NIME.

[4]  Roger B. Dannenberg,et al.  An On-Line Algorithm for Real-Time Accompaniment , 1984, ICMC.

[5]  Ali Taylan Cemgil,et al.  Bayesian Music Transcription , 1997 .

[6]  여운승.  International Conference on New Interfaces for Musical Expression(NIME) 2009 , 2009 .

[7]  George Tzanetakis,et al.  An experimental comparison of audio tempo induction algorithms , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[8]  Joseph T. Chung,et al.  Hyperinstruments: Musically Intelligent and Interactive Performance and Creativity Systems , 1989, ICMC.

[9]  François Pachet,et al.  The Continuator: Musical Interaction With Style , 2003, ICMC.

[10]  Gil Weinberg,et al.  Haile - an Interactive robotic percussionist , 2005, ICMC.

[11]  Eric Singer,et al.  Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), Montreal, Canada LEMUR GuitarBot: MIDI Robotic String Instrument , 2022 .

[12]  George Tzanetakis,et al.  MARSYAS: a framework for audio analysis , 1999, Organised Sound.

[13]  Gil Weinberg,et al.  Jam'aa - A Middle Eastern Percussion Ensemble for Human and Robotic Players , 2006, ICMC.

[14]  Robert Rowe,et al.  Machine Musicianship , 2001 .

[15]  Manjinder Singh Benning,et al.  Wireless inertial sensor package (WISP) , 2007, NIME '07.

[16]  Eric Singer,et al.  LEMUR's Musical Robots , 2004, NIME.

[17]  Richard A. Brown,et al.  Introduction to random signals and applied kalman filtering (3rd ed , 2012 .