Active Learning of Bayesian Probabilistic Movement Primitives

Learning from Demonstration permits non-expert users to easily and intuitively reprogram robots. Among approaches embracing this paradigm, probabilistic movement primitives (ProMPs) are a well-established and widely used method to learn trajectory distributions. However, providing or requesting useful demonstrations is not easy, as quantifying what constitutes a good demonstration in terms of generalization capabilities is not trivial. In this letter, we propose an active learning method for contextual ProMPs for addressing this problem. More specifically, we learn the trajectory distributions using a Bayesian Gaussian mixture model (BGMM) and then leverage the notion of epistemic uncertainties to iteratively choose new context query points for demonstrations. We show that this approach reduces the required number of human demonstrations. We demonstrate the effectiveness of the approach on a pouring task, both in simulation and on a real 7-DoF Franka Emika robot.

[1]  Aran Sena,et al.  Quantifying teaching behavior in robot learning from demonstration , 2019, Int. J. Robotics Res..

[2]  Jan Peters,et al.  Probabilistic movement primitives under unknown system dynamics , 2018, Adv. Robotics.

[3]  Ruzena Bajcsy,et al.  Active Learning for Vision-Based Robot Grasping , 2005, Machine Learning.

[4]  Artemy Kolchinsky,et al.  Estimating Mixture Entropy with Pairwise Distances , 2017, Entropy.

[5]  David Silver,et al.  Active learning from demonstration for robust autonomous navigation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[6]  Sylvain Calinon,et al.  Active Improvement of Control Policies with Bayesian Gaussian Mixture Model , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[8]  Jan Peters,et al.  Active Incremental Learning of Robot Movement Primitives , 2017, CoRL.

[9]  Bernhard Schölkopf,et al.  Using probabilistic movement primitives for striking movements , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[10]  A. Thomaz,et al.  Transparent active learning for robots , 2010, 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Manuela M. Veloso,et al.  Interactive Policy Learning through Confidence-Based Autonomy , 2014, J. Artif. Intell. Res..

[12]  Oliver Kroemer,et al.  Probabilistic movement primitives for coordination of multiple human–robot collaborative tasks , 2017, Auton. Robots.

[13]  Jan Peters,et al.  Incremental imitation learning of context-dependent motor skills , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[14]  Rajesh P. N. Rao,et al.  Active Imitation Learning , 2007, AAAI.

[15]  David D. Cox,et al.  Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms , 2013, SciPy.

[16]  Pierre-Yves Oudeyer,et al.  Learning to recognize objects through curiosity-driven manipulation with the iCub humanoid robot , 2013, 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[17]  Jürgen Schmidhuber,et al.  Optimal Artificial Curiosity, Creativity, Music, and the Fine Arts , 2005 .

[18]  Yuchen Zhao,et al.  Teaching Human Teachers to Teach Robot Learners , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[19]  David Cohn,et al.  Active Learning , 2010, Encyclopedia of Machine Learning.

[20]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[21]  Tucker Hermans,et al.  Active Learning of Probabilistic Movement Primitives , 2019, 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids).

[22]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[23]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[24]  Jan Peters,et al.  Using probabilistic movement primitives in robotics , 2017, Autonomous Robots.

[25]  Oliver Kroemer,et al.  Combining active learning and reactive control for robot grasping , 2010, Robotics Auton. Syst..