Phyletic evolution of neural feature detectors

In a quarter of a century of evolutionary computing, nature still seems to be teasing us with its complexity and flexibility whilst we struggle to apply our artificial creations, that perform so beautifully in blocks-world to the real world. We discuss some of the ways in which the biological world has seemed to defy the curse of dimensionality and present the results of an experiment to evolve neural network pattern detectors based on a pre-emptive 'phylogeny'. Strategies discussed are: congruent graduation of objective function and genome complexity; relaxation of objective function specificity; pre-evolved niche recombination; and fractal-like ontogenesis. A phyletic evolutionary architecture is proposed that combines these principles, together with three novel neural net transformations that preserve node-function integrity at different levels of complexity. Using a simple genetic algorithm, a number of 81-node fully recurrent neural nets were evolved to detect intermediate level features in 9/spl times/9 subimages. It is shown that by seeding the population with transformations of pre-evolved 3/spl times/3 detectors of constituent low-level features, evolution converged faster and to a more accurate and general solution than when they were evolved from a random population.

[1]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[2]  Philippe Collard,et al.  From GAs to artificial immune systems: improving adaptation in time dependent optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[3]  D. Parisi,et al.  Preadaptation in populations of neural networks evolving in a changing environment , 1995 .

[4]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[5]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[6]  Frédéric Gruau,et al.  Genetic Synthesis of Modular Neural Networks , 1993, ICGA.

[7]  Heinz Mühlenbein,et al.  Limitations of multi-layer perceptron networks-steps towards genetic neural networks , 1990, Parallel Comput..

[8]  Jeffrey Horn,et al.  Handbook of evolutionary computation , 1997 .

[9]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[10]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[11]  Hamid Bolouri,et al.  Staged training of Neocognitron by evolutionary algorithms , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[12]  Xin Yao,et al.  Evolutionary Artificial Neural Networks , 1993, Int. J. Neural Syst..

[13]  Risto Miikkulainen,et al.  Hierarchical evolution of neural networks , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[14]  Stephen F. Smith,et al.  The GENIE is out! (Who needs fitness to evolve?) , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[15]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[16]  Richard Walker,et al.  Niche Selection and the Evolution of Complex Behavior in a Changing EnvironmentA Simulation , 1999, Artificial Life.

[17]  Hiroaki Kitano.  A simple model of neurogenesis and cell differentiation based on evolutionary large-scale chaos , 1995 .

[18]  R. K. Ursem.  Multinational evolutionary algorithms , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).