Multi-modal Intention Prediction with Probabilistic Movement Primitives

This paper proposes a method for multi-modal prediction of intention based on a probabilistic description of movement primitives and goals. We target dyadic interaction between a human and a robot in a collaborative scenario. The robot acquires multi-modal models of collaborative action primitives containing gaze cues from the human partner and kinetic information about the manipulation primitives of its arm. We show that if the partner guides the robot with the gaze cue, the robot recognizes the intended action primitive even in the case of ambiguous actions. Furthermore, this prior knowledge acquired by gaze greatly improves the prediction of the future intended trajectory during a physical interaction. Results with the humanoid iCub are presented and discussed.

[1]  H. Kozima,et al.  A Robot that Learns to Communicate with Human Caregivers , 2001 .

[2]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[4]  Ana Paiva,et al.  Detecting user engagement with a robot companion using task and social interaction-based features , 2009, ICMI-MLMI '09.

[5]  A. Walker-Andrews.  Infants' perception of expressive behaviors: differentiation of multimodal information. , 1997, Psychological bulletin.

[6]  T. Nishida,et al.  Combining Multiple Types of Eye-gaze Information to Predict User ’ s Conversational Engagement , 2011 .

[7]  Joseph Kim,et al.  Collaborative Planning with Encoding of Users' High-Level Strategies , 2017, AAAI.

[8]  Ashwin P. Dani,et al.  Human intention inference through interacting multiple model filtering , 2015, 2015 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[9]  V. Javier Traver,et al.  Making service robots human-safe , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[10]  Matthias Vogelgesang,et al.  Multimodal integration of natural gaze behavior for intention recognition during object manipulation , 2009, ICMI-MLMI '09.

[11]  Helmut Prendinger,et al.  Eye movement as an indicator of users'involvement with embodied interfaces at the low level , 2005 .

[12]  Matthew W. Hoffman,et al.  A probabilistic model of gaze imitation and shared attention , 2006, Neural Networks.

[13]  Jianwei Zhang,et al.  Multimodal People Tracking and Trajectory Prediction based on Learned Generalized Motion Patterns , 2006, 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[14]  Siddhartha S. Srinivasa,et al.  Integrating human observer inferences into robot motion planning , 2014, Auton. Robots.

[15]  Gonzalo Ferrer,et al.  Bayesian Human Motion Intentionality Prediction in urban environments , 2014, Pattern Recognit. Lett..

[16]  François Charpillet,et al.  Prediction of Intention during Interaction with iCub with Probabilistic Movement Primitives , 2017, Front. Robot. AI.

[17]  Shumeet Baluja,et al.  Non-Intrusive Gaze Tracking Using Artificial Neural Networks , 1993, NIPS.

[18]  Masatsugu Kidode,et al.  Robot Navigation by Eye Pointing , 2005, ICEC.

[19]  Sofiane Boucenna,et al.  Evaluating the Engagement with Social Robots , 2015, International Journal of Social Robotics.

[20]  D. Muir,et al.  Gaze-following : its development and significance , 2007 .

[21]  Bernhard Schölkopf,et al.  Probabilistic Modeling of Human Movements for Intention Inference , 2012, Robotics: Science and Systems.

[22]  Erhardt Barth,et al.  Accurate Eye Centre Localisation by Means of Gradients , 2011, VISAPP.

[23]  Siddhartha S. Srinivasa,et al.  Generating Legible Motion , 2013, Robotics: Science and Systems.

[24]  Mohamed Chetouani,et al.  Towards Engagement Models that Consider Individual Factors in HRI: On the Relation of Extroversion and Negative Attitude Towards Robots to Gaze and Speech During a Human–Robot Assembly Task , 2015, Int. J. Soc. Robotics.

[25]  Philippe Gaussier,et al.  A Robot Learns the Facial Expressions Recognition and Face/Non-face Discrimination Through an Imitation Game , 2014, Int. J. Soc. Robotics.

[26]  Andrew N. Meltzoff,et al.  10 Eyes Wide Shut: The Importance of Eyes in Infant Gaze Following and Understanding Other Minds , 2006 .

[27]  Bilge Mutlu,et al.  Anticipatory robot control for efficient human-robot collaboration , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[28]  Rüdiger Dillmann,et al.  Armar II - a Learning and Cooperative Multimodal Humanoid Robot System , 2004, Int. J. Humanoid Robotics.

[29]  Ashwin P. Dani,et al.  Bayesian human intention inference through multiple model filtering with gaze-based priors , 2016, 2016 19th International Conference on Information Fusion (FUSION).