Some experimental evidence on the performance of GA-designed neural networks

While some efforts have been made to formalize the choice of parameters for artificial neural networks (multi-level perceptrons) using the backpropagation (BP) algorithm, studies reported in the literature to date appear to be dominated by use of heuristic methods of design. The objective of this paper is to test the comparative performance of neural networks with parameters determined by genetic algorithms (GAs). We have attempted to improve the quality of this type of research by comparing several models across seven real problem domains, including both regression and classification problems. Performance is benchmarked against neural networks whose parameters are selected using heuristic methods, as well as several widely used statistical models. For the problem domains tested, GA-assisted design yields models that consistently perform as well or better than heuristic methods and the statistical models.

[1]  James V. Hansen,et al.  Neural networks and traditional time series methods: a synergistic combination in state economic forecasts , 1997, IEEE Trans. Neural Networks.

[2]  James P. Ignizio,et al.  Boundary search procedure for the design and training of an ontogenic neural network classifier , 1998, Comput. Oper. Res..

[3]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[4]  Ronald H. Rasch,et al.  Factors Affecting Software Developers' Performance: An Integrated Approach , 1992, MIS Q..

[5]  Gary J. Koehler,et al.  Linear Discriminant Functions Determined by Genetic Search , 1991, INFORMS J. Comput..

[6]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[7]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[8]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[9]  A. Raftery Bayesian Model Selection in Social Research , 1995 .

[10]  Charles Leave Neural Networks: Algorithms, Applications and Programming Techniques , 1992 .

[11]  Marie Cottrell,et al.  Neural modeling for time series: A statistical stepwise method for weight elimination , 1995, IEEE Trans. Neural Networks.

[12]  Eduardo D. Sontag,et al.  Feedback Stabilization Using Two-Hidden-Layer Nets , 1991, 1991 American Control Conference.

[13]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[14]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[15]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[16]  Timothy Masters,et al.  Practical neural network recipes in C , 1993 .

[17]  James P. Ignizio,et al.  Boundary search procedure for the design and training of an ontogenic neural network classifier , 1998, Comput. Oper. Res..

[18]  B. L. Kalman,et al.  Why tanh: choosing a sigmoidal function , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[19]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[20]  David H. Wolpert,et al.  On the Connection between In-sample Testing and Generalization Error , 1992, Complex Syst..

[21]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[22]  Somnath Mukhopadhyay,et al.  A polynomial time algorithm for the construction and training of a class of multilayer perceptrons , 1993, Neural Networks.

[23]  R. E. Uhrig,et al.  Using genetic algorithms to select inputs for neural networks , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[24]  Wolfram Schiffmann,et al.  Application of Genetic Algorithms to the Construction of Topologies for Multilayer Perceptrons , 1993 .