Towards interactive physical robotic assistance: Parameterizing motion primitives through natural language

Natural language interaction between humans and robots is a very challenging topic, especially when it refers to motion descriptions in a certain environment. This problem is particularly relevant during physical human-robot interaction, e.g. in cooperative transportation tasks, where the partners' physical coupling requires an agreement on the way to follow. Understanding in depth the link between sentences, words, environmental properties and motions can deeply enhance the interaction between humans and robots. In this work, we propose a novel approach for learning relations and dependencies between motion, natural language and environmental properties using parameterized left-to-right time-based Hidden Markov Models. A natural language model represents the link between language and motion symbols while the HMMs parameterization corresponds to the explicit influence on motions of both words and environmental features. The proposed PHMM approach parameterizes the output and the transition probabilities using a non-linear dependency estimation. The method is validated by learning and generating navigation primitives in a 2 Degrees-Of-Freedom (DoF) virtual scenario.

[1]  Bodo Rosenhahn,et al.  Statistical and Geometrical Approaches to Visual Motion Analysis, International Dagstuhl Seminar, Dagstuhl Castle, Germany, July 13-18, 2008. Revised Papers , 2009, Lecture Notes in Computer Science.

[2]  Jun Tani,et al.  Learning Semantic Combinatoriality from the Interaction between Linguistic and Behavioral Processes , 2005, Adapt. Behav..

[3]  Hiroshi Ishiguro,et al.  Haptic Communication Between Humans and Robots , 2005, ISRR.

[4]  Volker Krüger,et al.  Recognition and Synthesis of Human Movements by Parametric HMMs , 2009, Statistical and Geometrical Approaches to Visual Motion Analysis.

[5]  Dongheui Lee,et al.  Incremental kinesthetic teaching of motion primitives using the motion refinement tube , 2011, Auton. Robots.

[6]  Statistical and Geometrical Approaches to Visual Motion Analysis, 13.07. - 18.07.2008 , 2008, Statistical and Geometrical Approaches to Visual Motion Analysis.

[7]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[8]  Katsu Yamane,et al.  Capture Database through Symbolization, Recognition and Generation of Motion Patterns , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[9]  Joris De Schutter,et al.  Invariant Description of Rigid Body Motion Trajectories , 2010 .

[10]  Yoshihiko Nakamura,et al.  Associative processes between behavioral symbols and a large scale language model , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Tetsuya Ogata,et al.  Human-Robot Cooperation using Quasi-symbols Generated by RNNPB Model , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[12]  SugitaYuuya,et al.  Learning Semantic Combinatoriality from the Interaction between Linguistic and Behavioral Processes , 2005 .

[13]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[15]  Katsu Yamane,et al.  Primitive communication based on motion recognition and generation with hierarchical mimesis model , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[16]  Satoshi Nakamura,et al.  Active learning of confidence measure function in robot language acquisition framework , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Yoshihiko Nakamura,et al.  Bigram-based natural language model and statistical motion symbol model for scalable language of humanoid robots , 2012, 2012 IEEE International Conference on Robotics and Automation.

[18]  Jeff A. Bilmes,et al.  A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models , 1998 .

[19]  Yoshihiko Nakamura,et al.  Statistically integrated semiotics that enables mutual inference between linguistic and behavioral symbols for humanoid robots , 2009, 2009 IEEE International Conference on Robotics and Automation.