Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity

The bootstrap problem is often recognized as one of the main challenges of evolutionary robotics: if all individuals from the first randomly generated population perform equally poorly, the evolutionary process won't generate any interesting solution. To overcome this lack of fitness gradient, we propose to efficiently explore behaviors until the evolutionary process finds an individual with a non-minimal fitness. To that aim, we introduce an original diversity-preservation mechanism, called behavioral diversity, that relies on a distance between behaviors (instead of genotypes or phenotypes) and multi-objective evolutionary optimization. This approach has been successfully tested and compared to a recently published incremental evolution method (multi-subgoal evolution) on the evolution of a neuro-controller for a light-seeking mobile robot. Results obtained with these two approaches are qualitatively similar although the introduced one is less directed than multi-subgoal evolution.

[1]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[2]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[3]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[4]  Inman Harvey,et al.  Seeing the light: artificial evolution, real vision , 1994 .

[5]  Stefano Nolfi,et al.  Evolving non-Trivial Behaviors on Real Robots: an Autonomous Robot that Picks up Objects , 1995, AI*IA.

[6]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[7]  Bruno Sareni,et al.  Fitness sharing and niching methods revisited , 1998, IEEE Trans. Evol. Comput..

[8]  Marco Colombetti,et al.  Incremental Robot Shaping , 1998, Connect. Sci..

[9]  Jean-Arcady Meyer,et al.  Evolution and development of neural controllers for locomotion, gradient-following, and obstacle-avoidance in artificial insects , 1998, IEEE Trans. Neural Networks.

[10]  Horst Bunke,et al.  A graph distance metric based on the maximal common subgraph , 1998, Pattern Recognit. Lett..

[11]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[12]  B. S. Manjunath,et al.  Incremental Evolution in Genetic Programming , 1998 .

[13]  Inman Harvey,et al.  Evolutionary Robotics: A Survey of Applications and Problems , 1998, EvoRobot.

[14]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[15]  D. Floreano,et al.  Incremental Evolution with Minimal Resources , 1999 .

[16]  Shan Jiang,et al.  Design of central pattern generator for humanoid robot walking based on multi-objective GA , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[17]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 , 2000 .

[18]  Dario Floreano,et al.  Evolution of Plastic Control Networks , 2001, Auton. Robots.

[19]  Gary B. Parker,et al.  The incremental evolution of gaits for hexapod robots , 2001 .

[20]  Jeffrey L. Krichmar,et al.  Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines , 2001, Complex..

[21]  Edwin D. de Jong,et al.  Reducing bloat and promoting diversity using multi-objective methods , 2001 .

[22]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[23]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[24]  J. Teo,et al.  Coordination and synchronization of locomotion in a virtual robot , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[25]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[26]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[27]  Hussein A. Abbass,et al.  Searching under Multi-evolutionary Pressures , 2003, EMO.

[28]  M. Walker.  Comparing the performance of incremental evolution to direct evolution , 2004 .

[29]  Simon Parsons,et al.  Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines by Stefano Nolfi and Dario Floreano, MIT Press, 320 pp., $28.00, ISBN 0-262-14070-5 , 2004, Knowledge engineering review (Print).

[30]  Hussein A. Abbass,et al.  Automatic Generation of Controllers for Embodied Legged Organisms: A Pareto Evolutionary Multi-Objective Approach , 2004, Evolutionary Computation.

[31]  Hussein A. Abbass,et al.  Multiobjective optimization for dynamic environments , 2005, 2005 IEEE Congress on Evolutionary Computation.

[32]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[33]  Jean-Arcady Meyer,et al.  Incremental Evolution of Target-Following Neuro-controllers for Flapping-Wing Animats , 2006, SAB.

[34]  Julian Togelius,et al.  Evolution of Neural Networks for Helicopter Control: Why Modularity Matters , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[35]  E de Margerie,et al.  Artificial evolution of the morphology and kinematics in a flapping-wing mini-UAV , 2007, Bioinspiration & biomimetics.

[36]  Peter J. Fleming,et al.  On the Evolutionary Optimization of Many Conflicting Objectives , 2007, IEEE Transactions on Evolutionary Computation.

[37]  Olivier Teytaud,et al.  On the Hardness of Offline Multi-objective Optimization , 2007, Evolutionary Computation.

[38]  Stéphane Doncieux,et al.  Incremental Evolution of Animats' Behaviors as a Multi-objective Optimization , 2008, SAB.

[39]  Stéphane Doncieux,et al.  MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars , 2008, Evol. Intell..

[40]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[41]  Kenneth O. Stanley,et al.  Exploiting Open-Endedness to Solve Problems Through the Search for Novelty , 2008, ALIFE.

[42]  Amiram Moshaiov,et al.  Multi-objective evolution of robot neuro-controllers , 2009, 2009 IEEE Congress on Evolutionary Computation.