Learning Walking Patterns for Kinematically Complex Robots Using Evolution Strategies

Manually developing walking patterns for kinematically complex robots can be a challenging and time-consuming task. In order to automate this design process, a learning system that generates, tests, and optimizes different walking patterns is needed, as well as the ability to accurately simulate a robot and its environment. In this work, we describe a learning system that uses the CMA-ES method from evolutionary computation to learn walking patterns for a complex legged robot. The robot's limbs are controlled using parametrized distorted sine waves, and the evolutionary algorithm optimizes the parameters of these waveforms, testing the walking patterns in a physical simulation. The best solutions evolved by this system has been transferred to and tested on a real robot, and has resulted in a gait that is superior to those previously designed by a human designer.

[1]  Hans-Paul Schwefel,et al.  Evolution and Optimum Seeking: The Sixth Generation , 1993 .

[2]  Martin R. Albrecht,et al.  ARAMIES: A FOUR-LEGGED CLIMBING AND WALKING ROBOT , 2005 .

[3]  Frank Kirchner,et al.  Exploiting Sensorimotor Coordination for Learning to Recognize Objects , 2007, IJCAI.

[4]  Phil Husbands,et al.  Evolution of central pattern generators for bipedal walking in a real-time physics environment , 2002, IEEE Trans. Evol. Comput..

[5]  Gregory S. Hornby,et al.  Autonomous evolution of gaits with the Sony Quadruped Robot , 1999 .

[6]  Frédéric Gruau,et al.  Cellular Encoding for interactive evolutionary robotics , 1996 .

[7]  Andrew H. Fagg,et al.  Genetic programming approach to the construction of a neural network for control of a walking robot , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[8]  Randall D. Beer,et al.  Biologically inspired approaches to robotics: what can we learn from insects? , 1997, CACM.

[9]  Kazuyuki Ito,et al.  A study of reinforcement learning for the robot with many degrees of freedom - acquisition of locomotion patterns for multi-legged robot , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[10]  Thomas Röfer,et al.  Evolutionary Gait-Optimization Using a Fitness Function Based on Proprioception , 2004, RoboCup.

[11]  Qijun Chen,et al.  Learning based gaits evolution for an AIBO dog , 2007, 2007 IEEE Congress on Evolutionary Computation.

[12]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[13]  Randall D. Beer,et al.  Evolving Dynamical Neural Networks for Adaptive Behavior , 1992, Adapt. Behav..

[14]  S. Grillner,et al.  Neural networks for vertebrate locomotion. , 1996, Scientific American.