Robot Learning with Task-Parameterized Generative Models

Task-parameterized models provide a representation of movement/behavior that can adapt to a set of task parameters describing the current situation encountered by the robot, such as location of objects or landmarks in its workspace. This paper gives an overview of the task-parameterized Gaussian mixture model (TP-GMM) presented in previous publications, and introduces a number of extensions and ongoing challenges required to move the approach toward unconstrained environments. In particular, it discusses its generalization capability and the handling of movements with a high number of degrees of freedom. It then shows that the method is not restricted to movements in task space, but that it can also be exploited to handle constraints in joint space, including priority constraints.

[1]  Danica Kragic,et al.  Learning Actions from Observations , 2010, IEEE Robotics & Automation Magazine.

[2]  Darwin G. Caldwell,et al.  On improving the extrapolation capability of task-parameterized movement models , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[4]  Heiga Zen,et al.  Reformulating the HMM as a trajectory model by imposing explicit relationships between static and dynamic feature vector sequences , 2007, Comput. Speech Lang..

[5]  Pieter Abbeel,et al.  Parameterized maneuver learning for autonomous helicopter flight , 2010, 2010 IEEE International Conference on Robotics and Automation.

[6]  Geoffrey E. Hinton,et al.  Deep Mixtures of Factor Analysers , 2012, ICML.

[7]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[8]  Satoshi Nakamura,et al.  Learning, Generation and Recognition of Motions by Reference-Point-Dependent Probabilistic Models , 2011, Adv. Robotics.

[9]  Jun Zhu,et al.  DP-space: Bayesian Nonparametric Subspace Clustering with Small-variance Asymptotics , 2015, ICML.

[10]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Jun Morimoto,et al.  On-line motion synthesis and adaptation using a trajectory database , 2012, Robotics Auton. Syst..

[12]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[13]  Darwin G. Caldwell,et al.  Learning from demonstrations with partially observable task parameters , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[15]  Richard Alan Peters,et al.  Superpositioning of behaviors learned through teleoperation , 2006, IEEE Transactions on Robotics.

[16]  Liqing Zhang,et al.  Kernelization of Tensor-Based Models for Multiway Data Analysis: Processing of Multidimensional Structured Data , 2013, IEEE Signal Processing Magazine.

[17]  Salah Bourennane,et al.  Denoising and Dimensionality Reduction Using Multilinear Tools for Hyperspectral Images , 2008, IEEE Geoscience and Remote Sensing Letters.

[18]  Darwin G. Caldwell,et al.  A task-parameterized probabilistic model with minimal intervention control , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Sethu Vijayakumar,et al.  Learning nullspace policies , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  A. Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[21]  Tamar Flash,et al.  Motor primitives in vertebrates and invertebrates , 2005, Current Opinion in Neurobiology.

[22]  Aude Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[23]  Jochen J. Steil,et al.  Interactive imitation learning of object movement skills , 2011, Autonomous Robots.

[24]  Mark J. F. Gales,et al.  Semi-tied covariance matrices for hidden Markov models , 1999, IEEE Trans. Speech Audio Process..

[25]  Carl E. Rasmussen,et al.  The Infinite Gaussian Mixture Model , 1999, NIPS.

[26]  Klas Kronander,et al.  Learning to control planar hitting motions in a minigolf-like task , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Fazel Naghdy,et al.  Learning Trajectories for Robot Programing by Demonstration Using a Coordinated Mixture of Factor Analyzers , 2016, IEEE Transactions on Cybernetics.

[28]  Yoshihiko Nakamura,et al.  Embodied Symbol Emergence Based on Mimesis Theory , 2004, Int. J. Robotics Res..

[29]  Johan A. K. Suykens,et al.  Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data , 2011, IEEE Signal Processing Letters.

[30]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[31]  Stefan Schaal,et al.  Incremental Online Learning in High Dimensions , 2005, Neural Computation.

[32]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[33]  Michael I. Jordan,et al.  Revisiting k-means: New Algorithms via Bayesian Nonparametrics , 2011, ICML.

[34]  Jun Morimoto,et al.  Learning parametric dynamic movement primitives from multiple demonstrations , 2011, Neural Networks.

[35]  Michael I. Jordan,et al.  Supervised learning from incomplete data via an EM approach , 1993, NIPS.

[36]  Nikolaos G. Tsagarakis,et al.  Statistical dynamical systems for skills acquisition in humanoids , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[37]  Peter J. Basser,et al.  A normal distribution for tensor-valued random variables: applications to diffusion tensor MRI , 2003, IEEE Transactions on Medical Imaging.

[38]  Aaron Hertzmann,et al.  Style machines , 2000, SIGGRAPH 2000.

[39]  Jan Peters,et al.  Nonamemanuscript No. (will be inserted by the editor) Reinforcement Learning to Adjust Parametrized Motor Primitives to , 2011 .

[40]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.

[41]  Charles Bouveyron,et al.  Model-based clustering of high-dimensional data: A review , 2014, Comput. Stat. Data Anal..

[42]  Sandra Hirche,et al.  Risk-Sensitive Optimal Feedback Control for Haptic Assistance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[43]  Brian Williams,et al.  Learning and Recognition of Hybrid Manipulation Motions in Variable Environments Using Probabilistic Flow Tubes , 2012, Int. J. Soc. Robotics.

[44]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[45]  Scott Niekum,et al.  Learning grounded finite-state representations from unstructured demonstrations , 2015, Int. J. Robotics Res..

[46]  Jochen J. Steil,et al.  A user study on kinesthetic teaching of redundant robots in task and configuration space , 2013, HRI 2013.

[47]  Olivier Stasse,et al.  Reverse Control for Humanoid Robot Task Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[48]  Geoffrey J. McLachlan,et al.  Modelling high-dimensional data by mixtures of factor analyzers , 2003, Comput. Stat. Data Anal..

[49]  Aude Billard,et al.  Statistical Learning by Imitation of Competing Constraints in Joint Space and Task Space , 2009, Adv. Robotics.

[50]  Junku Yuh,et al.  Underwater Robots , 2012, Springer Handbook of Robotics, 2nd Ed..

[51]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[52]  Matteo Saveriano,et al.  Incremental kinesthetic teaching of end-effector and null-space motion primitives , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[53]  Olivier Sigaud,et al.  Multiple task optimization using dynamical movement primitives for whole-body reactive control , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[54]  M. Hoagland,et al.  Feedback Systems An Introduction for Scientists and Engineers SECOND EDITION , 2015 .