Training oscillatory neural networks using natural gradient particle swarm optimization

In this paper, a new design of neural networks is introduced, which is able to generate oscillatory patterns in its output. The oscillatory neural network is used in a biped robot to enable it to learn to walk. The fundamental building block of the neural network proposed in this paper is O-neurons, which can generate oscillations in its transfer functions. O-neurons are connected and coupled with each other in order to shape a network, and their unknown parameters are found by a particle swarm optimization method. The main contribution of this paper is the learning algorithm that can combine natural policy gradient with particle swarm optimization methods. The oscillatory neural network has six outputs that determine set points for proportional-integral-derivative controllers in 6-DOF humanoid robots. Our experiment on the simulated humanoid robot presents smooth and flexible walking.

[1]  I A Basheer,et al.  Artificial neural networks: fundamentals, computing, design, and application. , 2000, Journal of microbiological methods.

[2]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[3]  Sten Grillner,et al.  Biological Pattern Generation: The Cellular and Computational Logic of Networks in Motion , 2006, Neuron.

[4]  Gary T. Anderson,et al.  Coupled Oscillator Control of Autonomous Mobile Robots , 2000, Auton. Robots.

[5]  J. J. Alcaraz-JimD,et al.  Motion Planning for Omnidirectional Dynamic Gait in Humanoid Soccer Robots , 2011 .

[6]  O. Kiehn,et al.  Physiological, anatomical and genetic identification of CPG neurons in the developing mammalian spinal cord , 2003, Progress in Neurobiology.

[7]  Andrej Gams,et al.  On-line learning and modulation of periodic movements with nonlinear dynamical systems , 2009, Auton. Robots.

[8]  Eric Chown,et al.  Omnidirectional Walking Using ZMP and Preview Control for the NAO Humanoid Robot , 2009, RoboCup.

[9]  F. Hackenberger.  Balancing Central Pattern Generator based Humanoid Robot Gait using Reinforcement Learning , 2007 .

[10]  Ludovic Righetti,et al.  Toward simple control for complex, autonomous robotic applications: combining discrete and rhythmic motor primitives , 2011, Auton. Robots.

[11]  A. Ijspeert,et al.  Dynamic hebbian learning in adaptive frequency oscillators , 2006 .

[12]  Alireza Akbarzadeh Tootoonchi,et al.  Online bio-inspired trajectory generation of seven-link biped robot based on T-S fuzzy system , 2014, Appl. Soft Comput..

[13]  Stefan Schaal,et al.  Natural Actor-Critic , 2003, Neurocomputing.

[14]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[15]  Ludovic Righetti,et al.  Programmable central pattern generators: an application to biped locomotion control , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[16]  Auke Jan Ijspeert,et al.  Central pattern generators for locomotion control in animals and robots: A review , 2008, Neural Networks.

[17]  J. Duysens,et al.  Neural control of locomotion; Part 1: The central pattern generator from cats to humans , 1998 .

[18]  Kamal Jamshidi,et al.  Modeling of mesencephalic locomotor region for Nao humanoid robot , 2012, Ind. Robot.

[19]  Stefan Schaal,et al.  Natural Actor-Critic , 2003, Neurocomputing.

[20]  Stefan Schaal,et al.  Policy Gradient Methods for Robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  E. Izhikevich.  Weakly Coupled Oscillators , 2006 .

[22]  Teresa Zielińska,et al.  Biological inspiration used for robots motion synthesis , 2009, Journal of Physiology-Paris.