Using behavioral exploration objectives to solve deceptive problems in neuro-evolution

Encouraging exploration, typically by preserving the diversity within the population, is one of the most common method to improve the behavior of evolutionary algorithms with deceptive fitness functions. Most of the published approaches to stimulate exploration rely on a distance between genotypes or phenotypes; however, such distances are difficult to compute when evolving neural networks due to (1) the algorithmic complexity of graph similarity measures, (2) the competing conventions problem and (3) the complexity of most neural-network encodings. In this paper, we introduce and compare two conceptually simple, yet efficient methods to improve exploration and avoid premature convergence when evolving both the topology and the parameters of neural networks. The two proposed methods, respectively called behavioral novelty and behavioral diversity, are built on multiobjective evolutionary algorithms and on a user-defined distance between behaviors. They can be employed with any genotype. We benchmarked them on the evolution of a neural network to compute a Boolean function with a deceptive fitness. The results obtained with the two proposed methods are statistically similar to those of NEAT and substantially better than those of the control experiment and of a phenotype-based diversity mechanism.

[1]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[2]  D. E. Goldberg,et al.  Simple Genetic Algorithms and the Minimal, Deceptive Problem , 1987 .

[3]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[4]  Kalyanmoy Deb,et al.  Massive Multimodality, Deception, and Genetic Algorithms , 1992, PPSN.

[5]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[6]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[7]  Frédéric Gruau,et al.  Automatic Definition of Modular Neural Networks , 1994, Adapt. Behav..

[8]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[9]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[10]  Horst Bunke,et al.  Recent developments in graph matching , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[11]  Richard A. Watson,et al.  Reducing Local Optima in Single-Objective Problems by Multi-objectivization , 2001, EMO.

[12]  Edwin D. de Jong,et al.  Reducing bloat and promoting diversity using multi-objective methods , 2001 .

[13]  B. E. Eckbo,et al.  Appendix , 1826, Epilepsy Research.

[14]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[15]  Graham Kendall,et al.  A Survey And Analysis Of Diversity Measures In Genetic Programming , 2002, GECCO.

[16]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[17]  Hussein A. Abbass,et al.  Searching under Multi-evolutionary Pressures , 2003, EMO.

[18]  Ernesto Benini,et al.  Genetic Diversity as an Objective in Multi-Objective Evolutionary Algorithms , 2003, Evolutionary Computation.

[19]  Daniel P. Lopresti,et al.  A fast technique for comparing graph representations with applications to performance evaluation , 2003, Document Analysis and Recognition.

[20]  Kalyanmoy Deb,et al.  Evaluating the -Domination Based Multi-Objective Evolutionary Algorithm for a Quick Computation of Pareto-Optimal Solutions , 2005, Evolutionary Computation.

[21]  Hussein A. Abbass,et al.  Multiobjective optimization for dynamic environments , 2005, 2005 IEEE Congress on Evolutionary Computation.

[22]  U. Alon,et al.  Spontaneous evolution of modularity and network motifs. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Jean-Arcady Meyer,et al.  Evolving PID-like Neurocontrollers for Nonlinear Control Problems , 2005, Control. Intell. Syst..

[24]  Peter J. Fleming,et al.  On the Evolutionary Optimization of Many Conflicting Objectives , 2007, IEEE Transactions on Evolutionary Computation.

[25]  Romain Raveaux,et al.  A Graph Classification Approach Using a Multi-objective Genetic Algorithm Application to Symbol Recognition , 2007, GbRPR.

[26]  Xin Yao,et al.  How well do multi-objective evolutionary algorithms scale to large problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[27]  Joshua D. Knowles,et al.  Multiobjectivization by Decomposition of Scalar Cost Functions , 2008, PPSN.

[28]  Stéphane Doncieux,et al.  MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars , 2008, Evol. Intell..

[29]  Kenneth O. Stanley,et al.  Exploiting Open-Endedness to Solve Problems Through the Search for Novelty , 2008, ALIFE.

[30]  Stéphane Doncieux,et al.  Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity , 2009, 2009 IEEE Congress on Evolutionary Computation.