Towards a synchronised Grammars framework for adaptive musical human-robot collaboration

We present an adaptive musical collaboration framework for interaction between a human and a robot. The aim of our work is to develop a system that receives feedback from the user in real time and learns the music progression style of the user over time. To tackle this problem, we represent a song as a hierarchically structured sequence of music primitives. By exploiting the sequential constraints of these primitives inferred from the structural information combined with user feedback, we show that a robot can play music in accordance with the user's anticipated actions. We use Stochastic Context-Free Grammars augmented with the knowledge of the learnt user's preferences. We provide synthetic experiments as well as a pilot study with a Baxter robot and a tangible music table. The synthetic results show the synchronisation and adaptivity features of our framework and the pilot study suggest these are applicable to create an effective musical collaboration experience.

[1]  Aaron F. Bobick,et al.  Recognition of Visual Activities and Interactions by Stochastic Parsing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  R. C. Underwood,et al.  Stochastic context-free grammars for tRNA modeling. , 1994, Nucleic acids research.

[3]  Tae-Kyun Kim,et al.  A syntactic approach to robot imitation learning using probabilistic activity grammars , 2013, Robotics Auton. Syst..

[4]  Sudhir Aggarwal,et al.  Password Cracking Using Probabilistic Context-Free Grammars , 2009, 2009 30th IEEE Symposium on Security and Privacy.

[5]  Jay Earley,et al.  An efficient context-free parsing algorithm , 1970, Commun. ACM.

[6]  Yiannis Demiris,et al.  Groovy Neural Networks , 2008, ECAI.

[7]  Andreas Stolcke,et al.  An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities , 1994, CL.

[8]  Cynthia Breazeal,et al.  Improved human-robot team performance using Chaski, A human-inspired plan execution system , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[9]  Ross Bencina,et al.  reacTIVision: a computer-vision framework for table-based tangible interaction , 2007, TEI.

[10]  Bob Pritchard,et al.  A TEMPORAL GENERATIVE GRAPH GRAMMAR FOR HARMONIC AND METRICAL STRUCTURE , 2014 .

[11]  Kaiqi Huang,et al.  An Extended Grammar System for Learning and Recognizing Complex Visual Events , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Marek P. Michalowski,et al.  A dancing robot for rhythmic social interaction , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[13]  Mason Bretan,et al.  Human-Robot Percussion Ensemble: Anticipation on the Basis of Visual Cues , 2013, IEEE Robotics & Automation Magazine.

[14]  Terrence Fong,et al.  Collaboration, Dialogue, Human-Robot Interaction , 2001, ISRR.

[15]  Brian Scassellati,et al.  Synchronization in Social Tasks: Robotic Drumming , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[16]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[17]  Kemal Ebcioglu,et al.  An Expert System for Chorale Harmonization , 1986, AAAI.

[18]  Hideki Koike,et al.  ImprovGenerator: Online Grammatical Induction for On-the-Fly Improvisation Accompaniment , 2010, NIME.

[19]  Cynthia Breazeal,et al.  Cost-Based Anticipatory Action Selection for Human–Robot Fluency , 2007, IEEE Transactions on Robotics.

[20]  Guy Hoffman,et al.  Gesture-based human-robot Jazz improvisation , 2010, 2010 IEEE International Conference on Robotics and Automation.

[21]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[22]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.