Online Goal Babbling for rapid bootstrapping of inverse models in high dimensions

We present a model of online Goal Babbling for the bootstrapping of sensorimotor coordination. By modeling infants' early goal-directed movements we show that inverse models can be bootstrapped within a few hundred movements even in very high-dimensional sensorimotor spaces. Our model thereby explains how infants might initially acquire reaching skills without the need for exhaustive exploration, and how robots can do so in a feasible way. We show that online learning in a closed loop with exploration allows substantial speed-ups and, in high-dimensional systems, outperforms previously published methods by orders of magnitude.

[1]  Pierre-Yves Oudeyer,et al.  Robust intrinsically motivated exploration and active learning , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[2]  Helge Ritter,et al.  Learning with the Self-Organizing Map , 1991 .

[3]  Y. Demiris,et al.  From motor babbling to hierarchical learning by imitation: a robot developmental pathway , 2005 .

[4]  Stephen Grossberg,et al.  Vector associative maps: Unsupervised real-time error-based learning and control of movement trajectories , 1991, Neural Networks.

[5]  Mitsuo Kawato,et al.  Feedback-Error-Learning Neural Network for Supervised Motor Learning , 1990 .

[6]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[7]  R. Johansson,et al.  Eye–Hand Coordination during Learning of a Novel Visuomotor Task , 2005, The Journal of Neuroscience.

[8]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[9]  G J Savelsbergh,et al.  The effect of posture on early reaching movements. , 1998, Journal of motor behavior.

[10]  E. Bushnell.  The decline of visually guided reaching during infancy , 1985 .

[11]  Jochen J. Steil,et al.  Mastering Growth while Bootstrapping Sensorimotor Coordination , 2010, EpiRob.

[12]  Stefan Schaal,et al.  Learning inverse kinematics , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[13]  Manuel Lopes,et al.  Body schema acquisition through active learning , 2010, 2010 IEEE International Conference on Robotics and Automation.

[14]  N. A. Bernshteĭn.  The co-ordination and regulation of movements , 1967 .

[15]  Stefan Schaal,et al.  Reinforcement learning by reward-weighted regression for operational space control , 2007, ICML '07.

[16]  C. Hofsten,et al.  Neonatal finger and arm movements as determined by a social and an object context , 1994 .

[17]  N. Berthier,et al.  Development of reaching in infancy , 2006, Experimental Brain Research.

[18]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[19]  Pierre-Yves Oudeyer,et al.  R-IAC : Robust Intrinsically Motivated Active Learning , 2009 .

[20]  Joachim Hoffmann,et al.  Exploiting redundancy for flexible behavior: unsupervised learning in a modular sensorimotor control architecture. , 2007, Psychological review.

[21]  J. Baily,et al.  Adaptation to Prisms: Do Proprioceptive Changes Mediate Adapted Behaviour with Ballistic Arm Movements? , 1972, The Quarterly journal of experimental psychology.

[22]  C. Hofsten.  Eye–hand coordination in the newborn. , 1982 .

[23]  Pierre-Yves Oudeyer,et al.  Maturationally-constrained competence-based intrinsically motivated learning , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[24]  Pierre-Yves Oudeyer,et al.  Intrinsically motivated goal exploration for active motor learning in robots: A case study , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Jochen J. Steil,et al.  Goal Babbling Permits Direct Learning of Inverse Kinematics , 2010, IEEE Transactions on Autonomous Mental Development.

[26]  S. Grossberg,et al.  A Self-Organizing Neural Model of Motor Equivalent Reaching and Tool Use by a Multijoint Arm , 1993, Journal of Cognitive Neuroscience.

[27]  Jochen J. Steil,et al.  Efficient exploration and learning of whole body kinematics , 2009, 2009 IEEE 8th International Conference on Development and Learning.