Genetic set recombination and its application to neural network topology optimisation

Forma analysis is applied to the task of optimising the connectivity of a feed- forward neural network with a single layer of hidden units. This problem is reformulated as a multiset optimisation problem, and techniques are developed to allow principled genetic search over fixed- and variable-size sets and multisets. These techniques require a further generalisation of the notion of gene, which is presented. The result is a non-redundant representation of the neural network topology optimisation problem, together with recombination operators which have carefully designed and well-understood properties. The techniques developed have relevance to the application of genetic algorithms to constrained optimisation problems.

[1]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[2]  David E. Goldberg,et al.  Alleles, loci and the traveling salesman problem , 1985 .

[3]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[4]  Geoffrey E. Hinton,et al.  Learning representations by back-propagation errors, nature , 1986 .

[5]  L. Darrell Whitley,et al.  Using Reproductive Evaluation to Improve Genetic Search and Heuristic Discovery , 1987, ICGA.

[6]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[7]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[8]  R.J.F. Dow,et al.  Neural net pruning-why and how , 1988, IEEE 1988 International Conference on Neural Networks.

[9]  Iain D. Craig.  Genetic Algorithms and Simulated Annealing edited by Lawrence Davis Pitman, London, 1987 (£19.95) , 1988, Robotica.

[10]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[11]  Rajarshi Das,et al.  A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization , 1989, ICGA.

[12]  L. D. Whitley,et al.  Scheduling Problems and Traveling Salesmen: The Genetic Edge Recombination Operator , 1989, ICGA.

[13]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[14]  Heinz Mühlenbein,et al.  Parallel Genetic Algorithms, Population Genetics, and Combinatorial Optimization , 1989, Parallelism, Learning, Evolution.

[15]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[16]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[17]  David E. Goldberg,et al.  Genetic Algorithms and Walsh Functions: Part I, A Gentle Introduction , 1989, Complex Syst..

[18]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[19]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[20]  R. Pfeifer,et al.  Connectionism in Perspective , 1989 .

[21]  L. Darrell Whitley,et al.  Genetic algorithms and neural networks: optimizing connections and connectivity , 1990, Parallel Comput..

[22]  Mike Rudnick,et al.  A bibliography of the intersection of genetic search and artificial neural networks , 1990 .

[23]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[24]  John R. Koza,et al.  Genetic programming: a paradigm for genetically breeding populations of computer programs to solve problems , 1990 .

[25]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .

[26]  Darrell Whitley,et al.  The Travelling Salesman and Sequence Scheduling: Quality Solutions using Genetic Edge Recombination , 1990 .

[27]  L. Darrell Whitley,et al.  Genetic Reinforcement Learning with Multilayer Neural Networks , 1991, ICGA.

[28]  Lashon B. Booker,et al.  Proceedings of the fourth international conference on Genetic algorithms , 1991 .

[29]  Lawrence Davis,et al.  Bit-Climbing, Representational Bias, and Test Suite Design , 1991, ICGA.

[30]  Nicholas J. Radcliffe,et al.  Equivalence Class Analysis of Genetic Algorithms , 1991, Complex Syst..

[31]  Michael D. Vose,et al.  Generalizing the Notion of Schema in Genetic Algorithms , 1991, Artif. Intell..

[32]  Larry J. Eshelman,et al.  On Crossover as an Evolutionarily Viable Strategy , 1991, ICGA.

[33]  Nicholas J. Radcliffe,et al.  Forma Analysis and Random Respectful Recombination , 1991, ICGA.

[34]  David E. Goldberg,et al.  Real-coded Genetic Algorithms, Virtual Alphabets, and Blocking , 1991, Complex Syst..

[35]  Gunar E. Liepins,et al.  Schema Disruption , 1991, ICGA.

[36]  Kenneth A. De Jong,et al.  On the Virtues of Parameterised Uniform Crossover , 1991, ICGA.

[37]  John R. Koza,et al.  Evolving a Computer Program to Generate Random Numbers Using the Genetic Programming Paradigm , 1991, ICGA.

[38]  Anthony N. Burkitt,et al.  Optimization of the Architecture of Feed-forward Neural Networks with Hidden Layers by Unit Elimination , 1991, Complex Syst..

[39]  Nicholas J. Radcliffe,et al.  Genetic neural networks on MIMD computers , 1992 .