Uncertainty-Aware Imitation Learning using Kernelized Movement Primitives

During the past few years, probabilistic approaches to imitation learning have earned a relevant place in the robotics literature. One of their most prominent features is that, in addition to extracting a mean trajectory from task demonstrations, they provide a variance estimation. The intuitive meaning of this variance, however, changes across different techniques, indicating either variability or uncertainty. In this paper we leverage kernelized movement primitives (KMP) to provide a new perspective on imitation learning by predicting variability, correlations and uncertainty using a single model. This rich set of information is used in combination with the fusion of optimal controllers to learn robot actions from data, with two main advantages: i) robots become safe when uncertain about their actions and ii) they are able to leverage partial demonstrations, given as elementary sub-tasks, to optimally perform a higher level, more complex task. We showcase our approach in a painting task, where a human user and a KUKA robot collaborate to paint a wooden board. The task is divided into two sub-tasks and we show that the robot becomes compliant (hence safe) outside the training regions and executes the two sub-tasks with optimal gains otherwise.

[1]  Wolfram Burgard,et al.  Most likely heteroscedastic Gaussian process regression , 2007, ICML '07.

[2]  Darwin G. Caldwell,et al.  Probabilistic Learning of Torque Controllers from Kinematic and Force Constraints , 2017, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Darwin G. Caldwell,et al.  Generalized Orientation Learning in Robot Task Space , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[4]  Sandra Hirche,et al.  Uncertainty-dependent optimal control for robot control considering high-order cost statistics , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Jan Peters,et al.  Bayesian optimization for learning gaits under uncertainty , 2015, Annals of Mathematics and Artificial Intelligence.

[6]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[7]  Sylvain Calinon,et al.  A tutorial on task-parameterized movement learning and retrieval , 2015, Intelligent Service Robotics.

[8]  Paul W. Goldberg,et al.  Regression with Input-dependent Noise: A Gaussian Process Treatment , 1997, NIPS.

[9]  Aude Billard,et al.  Statistical Learning by Imitation of Competing Constraints in Joint Space and Task Space , 2009, Adv. Robotics.

[10]  Kyungjae Lee,et al.  Uncertainty-Aware Learning from Demonstration Using Mixture Density Networks with Sampling-Free Variance Modeling , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[12]  Darwin G. Caldwell,et al.  Non-parametric Imitation Learning of Robot Motor Skills , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[13]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[14]  Darwin G. Caldwell,et al.  Learning optimal controllers in human-robot cooperative transportation tasks with position and force constraints , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[15]  Wolfram Burgard,et al.  Monte Carlo Localization: Efficient Position Estimation for Mobile Robots , 1999, AAAI/IAAI.

[16]  Darwin G. Caldwell,et al.  A task-parameterized probabilistic model with minimal intervention control , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Sandra Hirche,et al.  Bayesian uncertainty modeling for programming by demonstration , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Darwin G. Caldwell,et al.  Towards Minimal Intervention Control with Competing Constraints , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Darwin G. Caldwell,et al.  An Uncertainty-Aware Minimal Intervention Control Strategy Learned from Demonstrations , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Sandra Hirche,et al.  Risk-Sensitive Optimal Feedback Control for Haptic Assistance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[21]  Jochen J. Steil,et al.  Multiple task optimization with a mixture of controllers for motion generation , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Sergey Levine,et al.  Learning force-based manipulation of deformable objects from multiple demonstrations , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Darwin G. Caldwell,et al.  Kernelized movement primitives , 2017, Int. J. Robotics Res..

[24]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[25]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.