Comparative performance of backpropagation networks designed by genetic algorithms and heuristics

The design of neural network models involves numerous complexities, including the determination of input vectors, choosing the number of hidden layers and their computational units, and specifying activation functions for the latter. The combinatoric possibilities are daunting, yet experience has yielded informal guidelines that can be useful. Alternatively, current research on genetic algorithms (GA) suggests that they might be of practical use as a formal method of determining ‘good’ architectures for neural networks. In this paper, we use a genetic algorithm to find effective architectures for backpropagation neural networks (BP). We compare the performance of heuristically designed BP networks with that of GA-designed BP networks. Our test domains are sets of problems having compensatory, conjunctive, and mixed-decision structures. The results of our experiment suggest that heuristic methods produce architectures that are simpler and yet perform comparatively well.  1998 John Wiley & Sons, Ltd.

[1]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[2]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[3]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[4]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[5]  M.H. Hassoun,et al.  Fundamentals of Artificial Neural Networks , 1996, Proceedings of the IEEE.

[6]  B. L. Kalman,et al.  Why tanh: choosing a sigmoidal function , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[7]  Monica Adya,et al.  Does AI Research Aid Prediction? A Review and Evaluation , 1995, ICIS.

[8]  Timothy Masters,et al.  Practical neural network recipes in C , 1993 .

[9]  John W. Payne,et al.  Task complexity and contingent processing in decision making: An information search and protocol analysis☆ , 1976 .

[10]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[11]  Robert Libby,et al.  Accounting and human information processing : theory and applications , 1981 .

[12]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[13]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[14]  Sholom M. Weiss,et al.  An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods , 1989, IJCAI.

[15]  Michael T. Manry,et al.  Neural networks: Algorithms, applications, and programming techniques: By James A. Freeman and David M. Skapura, Addison-Wesley Publishing, Reading, MA, ISBN 0-201-51376-5 , 1994 .

[16]  Tariq Samad,et al.  Designing Application-Specific Neural Networks Using the Genetic Algorithm , 1989, NIPS.

[17]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[18]  Pat Langley,et al.  Elements of Machine Learning , 1995 .