Real time implementation of CTRNN and BPTT algorithm to learn on-line biped robot balance: Experiments on the standing posture

This paper describes experimental results regarding the real time implementation of continuous time recurrent neural networks (CTRNN) and the dynamic back-propagation through time (BPTT) algorithm for the on-line learning control laws. Experiments are carried out to control the balance of a biped robot prototype in its standing posture. The neural controller is trained to compensate for external perturbations by controlling the torso's joint motions. Algorithms are embedded in the real time electronic unit of the robot. On-line learning implementations are presented in detail. The results on learning behavior and control performance demonstrate the strength and the efficiency of the proposed approach.

[1]  Fethi Ben Ouezdou,et al.  Time Window Width Influence on Dynamic BPTT(h) Learning Algorithm Performances: Experimental Study , 2006, ICANN.

[2]  Tamás Szabó,et al.  Kernel CMAC with improved capability , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[3]  Jing Peng,et al.  An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories , 1990, Neural Computation.

[4]  Jason Noble,et al.  Homeostatic plasticity improves signal propagation in continuous-time recurrent neural networks , 2007, Biosyst..

[5]  A. PearlmutterB..  Gradient calculations for dynamic recurrent neural networks , 1995 .

[6]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[7]  Fethi Ben Ouezdou,et al.  Dynamic Recurrent Neural Network for Biped Robot Equilibrium Control: Preliminary Results , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[8]  Min-Jea Tahk,et al.  Real-time neural-network midcourse guidance , 2001 .

[9]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[10]  Atsuo Takanishi,et al.  Physical interaction between human and a bipedal humanoid robot-realization of human-follow walking , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[11]  B. Mohamed,et al.  Emulation of the Dynamic Effects of Human Torso During a Walking Gait , 2004 .

[12]  Hiroshi Shimizu,et al.  Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment , 1991, Biological Cybernetics.

[13]  Chih-Min Lin,et al.  Robust Fault-Tolerant Control for a Biped Robot Using a Recurrent Cerebellar Model Articulation Controller , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  Sang-Ho Hyon.  Compliant Terrain Adaptation for Biped Humanoids Without Measuring Ground Surface and Contact Forces , 2009, IEEE Transactions on Robotics.

[15]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[16]  Atsushi Konno,et al.  Design and development of the biped prototype ROBIAN , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[17]  Kohei Ohtsu,et al.  Minimum time ship maneuvering method using neural network and nonlinear model predictive compensator , 2004 .

[18]  Thierry Hoinville,et al.  Comparative study of two homeostatic mechanisms in evolved neural controllers for legged locomotion , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[19]  Ludovic Righetti,et al.  Programmable central pattern generators: an application to biped locomotion control , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[20]  Fethi Ben Ouezdou,et al.  Four DOF TORSO Dynamic Effects on Biped Walking Gait , 2002 .

[21]  Florentin Wörgötter,et al.  A Reflexive Neural Network for Dynamic Biped Walking Control , 2006, Neural Computation.

[22]  Jun Morimoto,et al.  Modulation of simple sinusoidal patterns by a coupled oscillator model for biped walking , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[23]  Anthony J. Robinson,et al.  Static and Dynamic Error Propagation Networks with Application to Speech Coding , 1987, NIPS.

[24]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[25]  Barak A. Pearlmutter.  Gradient calculations for dynamic recurrent neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[26]  Jun Morimoto,et al.  Learning from demonstration and adaptation of biped locomotion , 2004, Robotics Auton. Syst..

[27]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[28]  Randall D. Beer,et al.  Parameter Space Structure of Continuous-Time Recurrent Neural Networks , 2006, Neural Computation.

[29]  Zne-Jung Lee,et al.  Robust and fast learning for fuzzy cerebellar model articulation controllers , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[31]  Toshio Fukuda,et al.  Stabilization control of biped locomotion robot based learning with GAs having self-adaptive mutation and recurrent neural networks , 1997, Proceedings of International Conference on Robotics and Automation.

[32]  Chun-Fei Hsu.  Intelligent position tracking control for LCM drive using stable online self-constructing recurrent neural network controller with bound architecture , 2009 .

[33]  Miomir Vukobratovic,et al.  Zero-Moment Point - Thirty Five Years of its Life , 2004, Int. J. Humanoid Robotics.

[34]  Garrison W. Cottrell,et al.  Tau Net A neural network for modeling temporal variability , 1997, Neurocomputing.

[35]  Christophe Sabourin,et al.  Robustness of the dynamic walk of a biped robot subjected to disturbing external forces by using CMAC neural networks , 2005, Robotics Auton. Syst..

[36]  Alan F. Murray,et al.  Synaptic Rewiring for Topographic Map Formation , 2008, ICANN.

[37]  Meng Joo Er,et al.  Online adaptive fuzzy neural identification and control of a class of MIMO nonlinear systems , 2003, IEEE Trans. Fuzzy Syst..

[38]  Teodor Marcu,et al.  Design of fault detection for a hydraulic looper using dynamic neural networks , 2008 .

[39]  F. B. Ouezdou,et al.  Design and control of a 3DOFs parallel actuated mechanism for biped application , 2005 .

[40]  Aref Maalej,et al.  Simulations of the dynamic behavior of a bipedal robot with trunk and arms subjected to 3D external disturbances in a vertical posture, during walking and during object handling , 2009 .

[41]  David W. L. Wang,et al.  Feedforward and deterministic fuzzy control of balance and posture during human gait , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[42]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[43]  XuLei Yang,et al.  Robust Recurrent Neural Network Control of Biped Robot , 2007, J. Intell. Robotic Syst..

[44]  Andrew L. Kun,et al.  Adaptive dynamic balance of a biped robot using neural networks , 1996, Proceedings of IEEE International Conference on Robotics and Automation.