Confronting the challenge of learning a flexible neural controller for a diversity of morphologies

The ambulatory capabilities of legged robots offer the potential for access to dangerous and uneven terrain without a risk to human life. However, while machine learning has proven effective at training such robots to walk, a significant limitation of such approaches is that controllers trained for a specific robot are likely to fail when transferred to a robot with a slightly different morphology. This paper confronts this challenge with a novel strategy: Instead of training a controller for a particular quadruped morphology, it evolves a special function (through a method called HyperNEAT) that takes morphology as input and outputs an entire neural network controller fitted to the specific morphology. Once such a relationship is learned the output controllers are able to work on a diversity of different morphologies. Highlighting the unique potential of such an approach, in this paper a neural controller evolved for three different robot morphologies, which differ in the length of their legs, can interpolate to never-seen intermediate morphologies without any further training. Thus this work suggests a new research path towards learning controllers for whole ranges of morphologies: Instead of learning controllers themselves, it is possible to learn the relationship between morphology and control.

[1]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[2]  Phil Husbands,et al.  GasNets and other evovalble neural networks applied to bipedal locomotion , 2004 .

[3]  Masahiro Fujita,et al.  Evolving robust gaits with AIBO , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[4]  Rodney A. Brooks,et al.  A robot that walks; emergent behaviors from a carefully evolved network , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[5]  Cecilio Angulo,et al.  Evolving the Walking Behaviour of a 12 DOF Quadruped Using a Distributed Neural Architecture , 2006, BioADIT.

[6]  Phil Husbands,et al.  Evolution of central pattern generators for bipedal walking in a real-time physics environment , 2002, IEEE Trans. Evol. Comput..

[7]  S Grillner,et al.  Central pattern generators for locomotion, with special reference to vertebrates. , 1985, Annual review of neuroscience.

[8]  Kenneth O. Stanley,et al.  Autonomous Evolution of Topographic Regularities in Artificial Neural Networks , 2010, Neural Computation.

[9]  Aude Billard,et al.  GasNets and other Evolvable Neural Networks applied to Bipedal Locomotion , 2004 .

[10]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[11]  Josh Bongard,et al.  Evolving modular genetic regulatory networks , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[12]  Kenneth O. Stanley A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks , 2009 .

[13]  Kenneth O. Stanley,et al.  A Case Study on the Critical Role of Geometric Regularity in Machine Learning , 2008, AAAI.

[14]  Risto Miikkulainen,et al.  A Taxonomy for Artificial Embryogeny , 2003, Artificial Life.

[15]  Michiel van de Panne,et al.  Guided Optimization for Balanced Locomotion , 1995 .

[16]  K. Pearson The control of walking. , 1976, Scientific American.

[17]  Randall D. Beer,et al.  On the Dynamics of Small Continuous-Time Recurrent Neural Networks , 1995, Adapt. Behav..

[18]  Kenneth O. Stanley,et al.  Compositional Pattern Producing Networks : A Novel Abstraction of Development , 2007 .

[19]  Kenneth O. Stanley,et al.  Evolving Static Representations for Task Transfer , 2010, J. Mach. Learn. Res..

[20]  Kenneth O. Stanley,et al.  On the Performance of Indirect Encoding Across the Continuum of Regularity , 2011, IEEE Transactions on Evolutionary Computation.

[21]  Jimmy Secretan,et al.  Picbreeder: evolving pictures collaboratively online , 2008, CHI.

[22]  Shigenobu Kobayashi,et al.  Reinforcement learning of walking behavior for a four-legged robot , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[23]  Charles Ofria,et al.  Evolving coordinated quadruped gaits with the HyperNEAT generative encoding , 2009, 2009 IEEE Congress on Evolutionary Computation.

[24]  Randall D. Beer,et al.  The dynamics of adaptive behavior: A research program , 1997, Robotics Auton. Syst..

[25]  George A. Bekey,et al.  Gait Adaptation in a Quadruped Robot , 2002, Auton. Robots.

[26]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[27]  David Wettergreen,et al.  Gait Generation For Legged Robots , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Kenneth O. Stanley,et al.  A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks , 2009, Artificial Life.

[29]  Joel Lehman,et al.  Evolving policy geometry for scalable multiagent learning , 2010, AAMAS.