Evolutionary Design of Neural Networks

This thesis deals with methods for nding neural network architectures suitable for learning particular problems. We use an evolutionary algorithm with four di erent genetic encoding methods to search for the suitable architectures. We train the neural network weights with a separate neural learning algorithm. We use eight di erent learning problems for benchmarking the encoding methods. Four of the problems are arti cial (XOR, Encoder and two function approximation problems), three are real-world classi cation problems from the Proben1 benchmarking problem set, and one is a bankruptcy classi cation problem studied earlier in one of our projects. Our evaluation criteria are classi cation accuracy and e ciency for using only the relevant variables. The classi cation results are compared also to those for network architectures found by a systematic search.

[1]  A. Cangelosi,et al.  Cell division and migration in a 'genotype' for neural networks (Cell division and migration in neural networks) , 1993 .

[2]  F. Dellaert TOWARD A BIOLOGICALLY DEFENSIBLE MODEL OF DEVELOPMENT , 1995 .

[3]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[4]  Vasant Honavar,et al.  Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature , 1995 .

[5]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[6]  John Moody,et al.  Prediction Risk and Architecture Selection for Neural Networks , 1994 .

[7]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[8]  D. Parisi,et al.  “Genotypes” for neural networks , 1998 .

[9]  H. P. Schwefel,et al.  Numerische Optimierung von Computermodellen mittels der Evo-lutionsstrategie , 1977 .

[10]  Simon M. Lucas,et al.  A comparison of matrix rewriting versus direct encoding for evolving neural networks , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[11]  J. Elman,et al.  GENE REGULATION AND BIOLOGICAL DEVELOPMENT IN NEURAL NETWORKS : AN EXPLORATORY MODEL , 1995 .

[12]  Lutz Prechelt,et al.  A Set of Neural Network Benchmark Problems and Benchmarking Rules , 1994 .

[13]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[14]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[15]  Clive Richards,et al.  The Blind Watchmaker , 1987, Bristol Medico-Chirurgical Journal.

[16]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[17]  John R. Koza,et al.  Genetic programming: a paradigm for genetically breeding populations of computer programs to solve problems , 1990 .

[18]  PrecheltLutz Automatic early stopping using cross validation , 1998 .

[19]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[20]  Dipl. Ing. Karl Heinz Kellermayer NUMERISCHE OPTIMIERUNG VON COMPUTER-MODELLEN MITTELS DER EVOLUTIONSSTRATEGIE Hans-Paul Schwefel Birkhäuser, Basel and Stuttgart, 1977 370 pages Hardback SF/48 ISBN 3-7643-0876-1 , 1977 .

[21]  Frank Dellaert,et al.  Toward an evolvable model of development for autonomous agent synthesis , 1994 .

[22]  Robert M. Farber,et al.  How Neural Nets Work , 1987, NIPS.

[23]  David B. Fogel,et al.  Evolutionary algorithms in theory and practice , 1997, Complex.

[24]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[25]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[26]  Warren S. Sarle,et al.  Stopped Training and Other Remedies for Overfitting , 1995 .

[27]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[28]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[29]  Christopher M. Bishop,et al.  Regularization and complexity control in feed-forward networks , 1995 .

[30]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[31]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[32]  J. Fincham,et al.  At Home in the Universe: the Search for Laws of Complexity . By Stuart Kauffman. Viking 1995. viii + 321 pages. Hard cover. Price £20. ISBN 0 670 84735 6. , 1996 .

[33]  I. Aleksander Automata, languages, development , 1978 .

[34]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[35]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[36]  Jean-Arcady Meyer,et al.  Evolution and development of control architectures in animats , 1995, Robotics Auton. Syst..

[37]  Brad Fullmer and Risto Miikkulainen Using Marker-Based Genetic Encoding Of Neural Networks To Evolve Finite-State Behaviour , 1991 .

[38]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[39]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[40]  D. Parisi,et al.  Growing neural networks , 1991 .

[41]  Jeong-Sun Park Designing Neural Network Using Genetic Algorithm , 1997 .

[42]  J. Urgen Branke Evolutionary Algorithms for Neural Network Design and Training , 1995 .

[43]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[44]  Jari Vaario An emergent modeling method for artificial neural networks , 1994 .