Non-parametric Imitation Learning of Robot Motor Skills

Unstructured environments impose several challenges when robots are required to perform different tasks and adapt to unseen situations. In this context, a relevant problem arises: how can robots learn to perform various tasks and adapt to different conditions? A potential solution is to endow robots with learning capabilities. In this line, imitation learning emerges as an intuitive way to teach robots different motor skills. This learning approach typically mimics human demonstrations by extracting invariant motion patterns and subsequently applies these patterns to new situations. In this paper, we propose a novel kernel treatment of imitation learning, which endows the robot with imitative and adaptive capabilities. In particular, due to the kernel treatment, the proposed approach is capable of learning human skills associated with high-dimensional inputs. Furthermore, we study a new concept of correlation-adaptive imitation learning, which allows for the adaptation of correlations exhibited in high-dimensional demonstrated skills. Several toy examples and a collaborative task with a real robot are provided to verify the effectiveness of our approach.

[1]  Jan Peters,et al.  Demonstration based trajectory optimization for generalizable robot motions , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[2]  Darwin G. Caldwell,et al.  Generalized Task-Parameterized Skill Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Jan Peters,et al.  Reinforcement Learning to Adjust Robot Movements to New Situations , 2010, IJCAI.

[4]  Darwin G. Caldwell,et al.  Probabilistic Learning of Torque Controllers from Kinematic and Force Constraints , 2017, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Dana Kulic,et al.  Incremental learning of human behaviors using hierarchical hidden Markov models , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Jan Peters,et al.  Phase estimation for fast action recognition and trajectory generation in human–robot collaboration , 2017, Int. J. Robotics Res..

[7]  Wolfram Burgard,et al.  Most likely heteroscedastic Gaussian process regression , 2007, ICML '07.

[8]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[9]  Alexander G. Gray,et al.  Fast Nonparametric Conditional Density Estimation , 2007, UAI.

[10]  Byron Boots,et al.  Towards Robust Skill Generalization: Unifying Learning from Demonstration and Motion Planning , 2017, CoRL.

[11]  Darwin G. Caldwell,et al.  Hybrid Probabilistic Trajectory Optimization Using Null-Space Exploration , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Kimitoshi Yamazaki,et al.  Environment-adaptive interaction primitives through visual context for human–robot motor skill learning , 2018, Autonomous Robots.

[13]  Michael I. Jordan,et al.  Regression with input-dependent noise: A Gaussian process treatment , 1998 .

[14]  Jan Peters,et al.  Policy Search for Motor Primitives in Robotics , 2008, NIPS 2008.

[15]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[16]  Guy Lever,et al.  Modelling Policies in MDPs in Reproducing Kernel Hilbert Space , 2015, AISTATS.

[17]  Darwin G. Caldwell,et al.  Kernelized movement primitives , 2017, Int. J. Robotics Res..

[18]  Oliver Kroemer,et al.  Interaction primitives for human-robot cooperation tasks , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Olivier Sigaud,et al.  Robot Skill Learning: From Reinforcement Learning to Evolution Strategies , 2013, Paladyn J. Behav. Robotics.

[20]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[21]  Sylvain Calinon,et al.  A tutorial on task-parameterized movement learning and retrieval , 2015, Intelligent Service Robotics.

[22]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[23]  Byron Boots,et al.  Gaussian Process Motion planning , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Byron Boots,et al.  Motion Planning as Probabilistic Inference using Gaussian Processes and Factor Graphs , 2016, Robotics: Science and Systems.

[25]  Stefan Schaal,et al.  Learning variable impedance control , 2011, Int. J. Robotics Res..

[26]  Peters Jan,et al.  Jointly learning trajectory generation and hitting point prediction in robot table tennis , 2016 .

[27]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[28]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[29]  Darwin G. Caldwell,et al.  Learning bimanual end-effector poses from demonstrations using task-parameterized dynamical systems , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[31]  Jan Peters,et al.  Guiding Trajectory Optimization by Demonstrated Distributions , 2017, IEEE Robotics and Automation Letters.

[32]  Jochen J. Steil,et al.  Task-level imitation learning using variance-based movement optimization , 2009, 2009 IEEE International Conference on Robotics and Automation.

[33]  Sandra Hirche,et al.  Risk-Sensitive Optimal Feedback Control for Haptic Assistance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[34]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.