Kernelized movement primitives

Imitation learning has been studied widely as a convenient way to transfer human skills to robots. This learning approach is aimed at extracting relevant motion patterns from human demonstrations and subsequently applying these patterns to different situations. Despite the many advancements that have been achieved, solutions for coping with unpredicted situations (e.g., obstacles and external perturbations) and high-dimensional inputs are still largely absent. In this paper, we propose a novel kernelized movement primitive (KMP), which allows the robot to adapt the learned motor skills and fulfill a variety of additional constraints arising over the course of a task. Specifically, KMP is capable of learning trajectories associated with high-dimensional inputs owing to the kernel treatment, which in turn renders a model with fewer open parameters in contrast to methods that rely on basis functions. Moreover, we extend our approach by exploiting local trajectory representations in different coordinate systems that describe the task at hand, endowing KMP with reliable extrapolation capabilities in broader domains. We apply KMP to the learning of time-driven trajectories as a special case, where a compact parametric representation describing a trajectory and its first-order derivative is utilized. In order to verify the effectiveness of our method, several examples of trajectory modulations and extrapolations associated with time inputs, as well as trajectory adaptations with high-dimensional inputs are provided.

[1]  Oliver Kroemer,et al.  Interaction primitives for human-robot cooperation tasks , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Sylvain Calinon,et al.  Supervisory teleoperation with online learning and optimal control , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Jan Peters,et al.  Model-free Probabilistic Movement Primitives for physical interaction , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Jan Peters,et al.  Learning multiple collaborative tasks with a mixture of Interaction Primitives , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[7]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[8]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[9]  Edward Grant,et al.  Learning for Control , 2019 .

[10]  Byron Boots,et al.  Motion Planning as Probabilistic Inference using Gaussian Processes and Factor Graphs , 2016, Robotics: Science and Systems.

[11]  Andrej Gams,et al.  Coupling Movement Primitives: Interaction With the Environment and Bimanual Tasks , 2014, IEEE Transactions on Robotics.

[12]  Jan Peters,et al.  Demonstration based trajectory optimization for generalizable robot motions , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[13]  Jan Peters,et al.  Reinforcement Learning to Adjust Robot Movements to New Situations , 2010, IJCAI.

[14]  Stefan Schaal,et al.  Learning variable impedance control , 2011, Int. J. Robotics Res..

[15]  Byron Boots,et al.  Towards Robust Skill Generalization: Unifying Learning from Demonstration and Motion Planning , 2017, CoRL.

[16]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[17]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[18]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[19]  Darwin G. Caldwell,et al.  Learning Competing Constraints and Task Priorities from Demonstrations of Bimanual Skills , 2017, ArXiv.

[20]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[21]  Affan Pervez,et al.  Learning task-parameterized dynamic movement primitives using mixture of GMMs , 2018, Intell. Serv. Robotics.

[22]  Sergey Levine,et al.  Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics , 2014, NIPS.

[23]  Jan Peters,et al.  Phase estimation for fast action recognition and trajectory generation in human–robot collaboration , 2017, Int. J. Robotics Res..

[24]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[25]  Peter Englert,et al.  Probabilistic model-based imitation learning , 2013, Adapt. Behav..

[26]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[27]  Darwin G. Caldwell,et al.  Robot motor skill coordination with EM-based Reinforcement Learning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Darwin G. Caldwell,et al.  Generalized Task-Parameterized Skill Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[29]  Sylvain Calinon,et al.  A tutorial on task-parameterized movement learning and retrieval , 2016, Intell. Serv. Robotics.

[30]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[31]  Darwin G. Caldwell,et al.  Generalized Task-Parameterized Movement Primitives , 2017, ArXiv.

[32]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[33]  Sandra Hirche,et al.  Risk-Sensitive Optimal Feedback Control for Haptic Assistance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[34]  Darwin G. Caldwell,et al.  Learning optimal controllers in human-robot cooperative transportation tasks with position and force constraints , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Michael Rabadi,et al.  Kernel Methods for Machine Learning , 2015 .

[36]  Bernhard Schölkopf,et al.  Jointly learning trajectory generation and hitting point prediction in robot table tennis , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[37]  Brett R Fajen,et al.  Behavioral dynamics of steering, obstacle avoidance, and route selection. , 2003, Journal of experimental psychology. Human perception and performance.

[38]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[39]  Jan Peters,et al.  Using probabilistic movement primitives in robotics , 2017, Autonomous Robots.

[40]  Stefan Schaal,et al.  Natural Actor-Critic , 2003, Neurocomputing.

[41]  Olivier Sigaud,et al.  Robot Skill Learning: From Reinforcement Learning to Evolution Strategies , 2013, Paladyn J. Behav. Robotics.

[42]  Aude Billard,et al.  Reinforcement learning for imitating constrained reaching movements , 2007, Adv. Robotics.

[43]  Carme Torras,et al.  A robot learning from demonstration framework to perform force-based manipulation tasks , 2013, Intelligent Service Robotics.

[44]  Sergey Levine,et al.  PLATO: Policy learning using adaptive trajectory optimization , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[45]  Jan Peters,et al.  Guiding Trajectory Optimization by Demonstrated Distributions , 2017, IEEE Robotics and Automation Letters.

[46]  Jochen J. Steil,et al.  Task-level imitation learning using variance-based movement optimization , 2009, 2009 IEEE International Conference on Robotics and Automation.

[47]  Jan Peters,et al.  Policy Search for Motor Primitives in Robotics , 2008, NIPS 2008.

[48]  Jun Morimoto,et al.  Learning Stylistic Dynamic Movement Primitives from multiple demonstrations , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[49]  Jun Morimoto,et al.  Learning from demonstration and adaptation of biped locomotion , 2004, Robotics Auton. Syst..