Reaching movement generation with a recurrent neural network based on learning inverse kinematics for the humanoid robot iCub

We present a dynamical system approach that couples task and joint space by means of an attractor-based content addressable memory. The respective recurrent reservoir network simultaneously provides a novel control framework for goal directed movement generation. The network first learns to associate end effector coordinates with joint angles by means of reservoir attractor states and thereby implements forward and inverse kinematics. Generalization of the learned kinematics to a wide range of untrained target positions is achieved by modulating the attractors with desired input states. We show that this representation of the static kinematic mapping within a dynamical system also enables smooth trajectory generation by exploiting the transient network dynamics when approaching an attractor state. A further strength of the proposed approach is that efficient online learning and execution of the network makes it real-time capable. We demonstrate the network's generalization abilities and evaluate controller properties systematically for arm movements of the humanoid robot iCub.

[1]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[2]  Jun Tani,et al.  On-line Imitative Interaction with a Humanoid Robot Using a Dynamic Neural Network Model of a Mirror System , 2004, Adapt. Behav..

[3]  Tadashi Yamazaki,et al.  The cerebellum as a liquid state machine , 2007, Neural Networks.

[4]  Jochen J. Steil,et al.  Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning , 2007, Neural Networks.

[5]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[6]  J.J. Steil,et al.  Backpropagation-decorrelation: online recurrent learning with O(N) complexity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[7]  Benjamin Schrauwen,et al.  Generative Modeling of Autonomous Robots and their Environments using Reservoir Computing , 2007, Neural Processing Letters.

[8]  Jochen J. Steil,et al.  Efficient exploration and learning of whole body kinematics , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[9]  Jun Nakanishi,et al.  Learning rhythmic movements by demonstration using nonlinear oscillators , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  R.F. Reinhart,et al.  Recurrent Neural Associative Learning of Forward and Inverse Kinematics for Movement Generation of the Redundant PA-10 Robot , 2008, 2008 ECSIS Symposium on Learning and Adaptive Behaviors for Robotic Systems (LAB-RS).

[11]  A. Meltzoff,et al.  The Robot in the Crib: A Developmental Analysis of Imitation Skills in Infants and Robots. , 2008, Infant and child development.

[12]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[13]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[14]  Masao Ito,et al.  Historical Review of the Significance of the Cerebellum and the Role of Purkinje Cells in Motor Learning , 2002, Annals of the New York Academy of Sciences.

[15]  Jun Tani,et al.  Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB , 2004, Neural Networks.

[16]  Jun Tani,et al.  Codevelopmental Learning Between Human and Humanoid Robot Using a Dynamic Neural-Network Model , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[18]  Wolfgang Maass,et al.  Movement Generation with Circuits of Spiking Neurons , 2005, Neural Computation.

[19]  Jochen J. Steil,et al.  Attractor-based computation with reservoirs for online learning of inverse kinematics , 2009, ESANN.

[20]  Brian Scassellati,et al.  A Fast and Efficient Model for Learning to Reach , 2005, Int. J. Humanoid Robotics.

[21]  Stefan Schaal,et al.  Learning inverse kinematics , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[22]  Giulio Sandini,et al.  The iCub humanoid robot: an open platform for research in embodied cognition , 2008, PerMIS.

[23]  Jochen Triesch,et al.  A Gradient Rule for the Plasticity of a Neuron's Intrinsic Excitability , 2005, ICANN.

[24]  Stefan Schaal,et al.  Computational approaches to motor learning by imitation. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[25]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[26]  Herbert Jaeger,et al.  Adaptive Nonlinear System Identification with Echo State Networks , 2002, NIPS.

[27]  Eduardo D. Sontag,et al.  Principles of real-time computing with feedback applied to cortical microcircuit models , 2005, NIPS.

[28]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .