Music-Ensemble Robot That Is Capable of Playing the Theremin While Listening to the Accompanied Music

Our goal is to achieve a musical ensemble among a robot and human musicians where the robot listens to the music with its own microphones. The main issues are (1) robust beat-tracking since the robot hears its own generated sounds in addition to the accompanied music, and (2) robust synchronizing its performance with the accompanied music even if humans' musical performance fluctuates. This paper presents a music-ensemble Thereminist robot implemented on the humanoid HRP- 2 with the following three functions: (1) self-generated Theremin sound suppression by semi-blind Independent Component Analysis, (2) beat tracking robust against tempo fluctuation in humans' performance, and (3) feedforward control of Theremin pitch. Experimental results with a human drummer show the capability of this robot for the adaptation to the temporal fluctuation in his performance.

[1]  Gil Weinberg,et al.  Toward Robotic Musicianship , 2006, Computer Music Journal.

[2]  A. Glinsky,et al.  The Theremin in the Emergence of Electronic Music , 1992 .

[3]  K. Kawamura,et al.  Music Playing Robot , 1999 .

[4]  Tetsuya Ogata,et al.  Thereminist robot: Development of a robot theremin player with feedforward and feedback arm control based on a Theremin's pitch model , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Gil Weinberg,et al.  The interactive robotic percussionist - new developments in form, mechanics, perception and interaction design , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[6]  Tetsuya Ogata,et al.  A robot listens to music and counts its beats aloud by separating music from counting voice , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Christopher Raphael,et al.  Music score alignment and computer accompaniment , 2006, CACM.

[8]  Fumio Kanehiro,et al.  Humanoid robot HRP-2 , 2008, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[9]  Tetsuya Ogata,et al.  Barge-in-able robot audition based on ICA and missing feature theory under semi-blind situation , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Atsuo Takanishi,et al.  Development of a aural real-time rhythmical and harmonic tracking to enable the musical interaction with the Waseda Flutist Robot , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Kazuyoshi Yoshii,et al.  A robot uses its own microphone to synchronize its steps to musical beats while scatting and singing , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.