Training Neural Networks with GA Hybrid Algorithms

Training neural networks is a complex task of great impor- tance in the supervised learning field of research. In this work we tackle this problem with five algorithms, and try to offer a set of results that could hopefully foster future comparisons by following a kind of stan- dard evaluation of the results (the Prechelt approach). To achieve our goal of studying in the same paper population based, local search, and hybrid algorithms, we have selected two gradient descent algorithms: Backpropagation and Levenberg-Marquardt, one population based heu- ristic such as a Genetic Algorithm, and two hybrid algorithms combining this last with the former local search ones. Our benchmark is composed of problems arising in Medicine, and our conclusions clearly establish the advantages of the proposed hybrids over the pure algorithms.

[1]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[2]  Pat Langley,et al.  Models of Incremental Concept Formation , 1990, Artif. Intell..

[3]  Lutz Prechelt,et al.  PROBEN 1 - a set of benchmarks and benchmarking rules for neural network training algorithms , 1994 .

[4]  Enrique Alba,et al.  Full Automatic ANN Design: A Genetic Approach , 1993, IWANN.

[5]  O. Mangasarian,et al.  Pattern Recognition Via Linear Programming: Theory and Application to Medical Diagnosis , 1989 .

[6]  O. Mangasarian,et al.  Multisurface method of pattern separation for medical diagnosis applied to breast cytology. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Walker H. Land,et al.  Breast cancer screening using evolved neural networks , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Jarmo T. Alander,et al.  An Indexed Bibliography of Genetic Algorithms , 1995 .

[10]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[11]  Pedro Larrañaga,et al.  Adjusting Weights in Artificial Neural Networks using Evolutionary Algorithms , 2002, Estimation of Distribution Algorithms.

[12]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[13]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[14]  O. Mangasarian,et al.  Robust linear programming discrimination of two linearly inseparable sets , 1992 .

[15]  Moonis Ali,et al.  Methodology and Tools in Knowledge-Based Systems , 1998, Lecture Notes in Computer Science.

[16]  Carlos Cotta,et al.  On Decision-Making in Strong Hybrid Evolutionary Algorithms , 1998, IEA/AIE.

[17]  Alberto Prieto,et al.  New Trends in Neural Computation , 1993 .

[18]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[19]  Richard S. Johannes,et al.  Using the ADAP Learning Algorithm to Forecast the Onset of Diabetes Mellitus , 1988 .

[20]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[21]  Werner Erhard,et al.  The Improvement and Comparison of Different Algorithms for Optimizing Neural Networks on the MasPar MP-2 , 1998, NC.

[22]  Thomas F. Coleman,et al.  Large-Scale Numerical Optimization , 1990 .

[23]  R. Detrano,et al.  International application of a new probability algorithm for the diagnosis of coronary artery disease. , 1989, The American journal of cardiology.

[24]  Erick Cantú-Paz.  Pruning Neural Networks with Distribution Estimation Algorithms , 2003, GECCO.

[25]  Thomas Ragg,et al.  Automatic determination of optimal network topologies based on information theory and evolution , 1997, EUROMICRO 97. Proceedings of the 23rd EUROMICRO Conference: New Frontiers of Information Technology (Cat. No.97TB100167).

[26]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[27]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[28]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.