Training neural networks by means of genetic algorithms working on very long chromosomes

In the neural network/genetic algorithm community, rather limited success in the training of neural networks by genetic algorithms has been reported. In a paper by Whitley et al. (1991), he claims that, due to "the multiple representations problem", genetic algorithms will not effectively be able to train multilayer perceptrons, whose chromosomal representation of its weights exceeds 300 bits. In the following paper, by use of a "real-life problem", known to be non-trivial, and by a comparison with "classic" neural net training methods, I will try to show, that the modest success of applying genetic algorithms to the training of perceptrons, is caused not so much by the "multiple representations problems" as by the fact that problem-specific knowledge available is often ignored, thus making the problem unnecessarily tough for the genetic algorithm to solve. Special success is obtained by the use of a new fitness function, which takes into account the fact that the search performed by a genetic algorithm is holistic, and not local as is usually the case when perceptrons are trained by traditional methods.

[1]  Scott E. Fahlman,et al.  The Recurrent Cascade-Correlation Architecture , 1990, NIPS.

[2]  James E. Baker,et al.  Adaptive Selection Methods for Genetic Algorithms , 1985, International Conference on Genetic Algorithms.

[3]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[4]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[5]  Thomas P. Caudell,et al.  Parametric Connectivity: Training of Constrained Networks using Genetic Algorithms , 1989, ICGA.

[6]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[7]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[8]  Esther Levin,et al.  Accelerated Learning in Layered Neural Networks , 1988, Complex Syst..

[9]  Byoung-Tak Zhang,et al.  Genetic Programming of Minimal Neural Nets Using Occam's Razor , 1993, ICGA.

[10]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms Revisited: Studies in Mixed Size and Scale , 1990, Complex Syst..

[11]  In Schoenauer,et al.  Parallel Problem Solving from Nature , 1990, Lecture Notes in Computer Science.

[12]  Kalyanmoy Deb,et al.  Don't Worry, Be Messy , 1991, ICGA.

[13]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[14]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[15]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[16]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[17]  Larry J. Eshelman,et al.  Biases in the Crossover Landscape , 1989, ICGA.

[18]  L. Darrell Whitley,et al.  Genetic Reinforcement Learning with Multilayer Neural Networks , 1991, ICGA.

[19]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[20]  David H. Ackley,et al.  An empirical study of bit vector function optimization , 1987 .

[21]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[22]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[23]  Lawrence Davis,et al.  Adapting Operator Probabilities in Genetic Algorithms , 1989, ICGA.

[24]  András Márkus Dual Insights into Genetic Algorithms , 1993, ICGA.

[25]  Steve G. Romaniuk Evolutionary Growth Perceptrons , 1993, ICGA.

[26]  David E. Goldberg,et al.  Genetic Algorithm Difficulty and the Modality of Fitness Landscapes , 1994, FOGA.

[27]  Gary B. Lamont,et al.  Comparison of Parallel Messy Genetic Algorithm Data Distribution Strategies , 1993, ICGA.

[28]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[29]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[30]  Marc Schoenauer,et al.  Genetic Lander: An Experiment in Accurate Neuro-Genetic Control , 1994, PPSN.