Learning adaptive movements from demonstration and self-guided exploration

The combination of imitation and exploration strategies is used in this paper to transfer sensory-motor skills to robotic platforms. The aim is to be able to learn very different tasks with good generalization capabilities and starting from a few demonstrations. This goal is achieved by learning a task-parameterized model from demonstrations where a teacher shows the task corresponding to different possible values of preassigned parameters. In this manner, new reproductions can be generated for new situations by assigning new values to the parameters, thus achieving very precise generalization capabilities. In this paper we propose a novel algorithm that is able to learn the model together with its dependence from the task-parameters, without specifying a predefined relationship or structure. The algorithm is able to learn the model starting from a few demonstrations by applying an exploration strategy that refines the learnt model autonomously. The algorithm is tested on a reaching task performed with a Barrett WAM manipulator.

[1]  Darwin G. Caldwell,et al.  Bayesian Nonparametric Multi-Optima Policy Search in Reinforcement Learning , 2013, AAAI.

[2]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[3]  Michael Gasser,et al.  The Development of Embodied Cognition: Six Lessons from Babies , 2005, Artificial Life.

[4]  Jun Morimoto,et al.  Learning parametric dynamic movement primitives from multiple demonstrations , 2011, Neural Networks.

[5]  Pierre-Yves Oudeyer,et al.  Self-organization of early vocal development in infants and machines: the role of intrinsic motivation , 2014, Front. Psychol..

[6]  J. Konczak,et al.  The development toward stereotypic arm kinematics during reaching in the first 3 years of life , 1997, Experimental Brain Research.

[7]  Olivier Sigaud,et al.  Learning compact parameterized skills with a single regression , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[8]  Peter I. Corke,et al.  MATLAB toolboxes: robotics and vision for students and teachers , 2007, IEEE Robotics & Automation Magazine.

[9]  Peter J. Basser,et al.  Spectral decomposition of a 4th-order covariance tensor: Applications to diffusion tensor MRI , 2007, Signal Process..

[10]  Bruno Castro da Silva,et al.  Learning parameterized motor skills on a humanoid robot , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Pierre-Yves Oudeyer,et al.  Properties for efficient demonstrations to a socially guided intrinsically motivated learner , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[12]  Ales Ude,et al.  Motion imitation and recognition using parametric hidden Markov models , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[13]  Darwin G. Caldwell,et al.  On improving the extrapolation capability of task-parameterized movement models , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Michael I. Jordan,et al.  Supervised learning from incomplete data via an EM approach , 1993, NIPS.

[15]  Nikolaos G. Tsagarakis,et al.  Statistical dynamical systems for skills acquisition in humanoids , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[16]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.