An empirical study of design and testing of hybrid evolutionary–neural approach for classification

We propose a hybrid evolutionary-neural approach for binary classification that incorporates a special training data over-fitting minimizing selection procedure for improving the prediction accuracy on holdout sample. Our approach integrates parallel global search capability of genetic algorithms (GAs) and local gradient-descent search of the back-propagation algorithm. Using a set of simulated and real life data sets, we illustrate that the proposed hybrid approach fares well, both in training and holdout samples, when compared to the traditional back-propagation artificial neural network (ANN) and a genetic algorithm-based artificial neural network (GA-ANN).

[1]  Larry R. Medsker,et al.  Models and Guidelines for Integrating Expert Systems and Neural Networks , 1992 .

[2]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[3]  Hans-Georg Beyer.  On the "Explorative Power" of ES/EP-like Algorithms , 1998, Evolutionary Programming.

[4]  Volker Nissen,et al.  On the robustness of population-based versus point-based optimization in the presence of noise , 1998, IEEE Trans. Evol. Comput..

[5]  Cihan H. Dagli.  Artificial Neural Networks for Intelligent Manufacturing , 1994 .

[6]  Gregory R. Madey,et al.  Hybrid intelligent systems: Tools for decision making in intelligent manufacturing , 1994 .

[7]  Michael Lee Steib,et al.  Expert systems for guiding backpropagation training of layered perceptrons , 1991 .

[8]  Ming S. Hung,et al.  Training neural networks with the GRG2 nonlinear optimizer , 1993 .

[9]  L. C. Rabelo,et al.  Synergy of artificial neural networks and knowledge-based expert systems for intelligent FMS scheduling , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[10]  Parag C. Pendharkar,et al.  The Wisconsin Division of Narcotics Enforcement Uses Multi-Agent Information Systems to Investigate Drug Crimes , 1999, Interfaces.

[11]  Stewart W. Wilson.  GA-Easy Does Not Imply Steepest-Ascent Optimizable , 1991, ICGA.

[12]  Chris Nikolopoulos,et al.  A hybrid expert system for investment advising , 1994 .

[13]  B. Curry,et al.  Neural networks: a need for caution , 1997 .

[14]  Lashon B. Booker,et al.  Proceedings of the fourth international conference on Genetic algorithms , 1991 .

[15]  Efraim Turban,et al.  Integrating Expert Systems and Neural Computing for Decision Support , 1994, HICSS.

[16]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[17]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[18]  Gary J. Koehler,et al.  Linear Discriminant Functions Determined by Genetic Search , 1991, INFORMS J. Comput..

[19]  L. Darrell Whitley,et al.  Optimizing Neural Networks Using FasterMore Accurate Genetic Search , 1989, ICGA.

[20]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[21]  Larry R. Medsker,et al.  Hybrid Intelligent Systems , 1995, Springer US.

[22]  Randall S. Sexton,et al.  Optimization of neural networks: A comparative analysis of the genetic algorithm and simulated annealing , 1999, Eur. J. Oper. Res..

[23]  Prakash L. Abad,et al.  New LP based heuristics for the classification problem , 1993 .

[24]  Gary J. Koehler,et al.  Minimizing Misclassifications in Linear Discriminant Analysis , 1990 .

[25]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[26]  Gregory R. Madey,et al.  The Design and Validation of a Hybrid Information System for the Auditor's Going Concern Decision , 1998, J. Manag. Inf. Syst..

[27]  Bahram Alidaee,et al.  Global optimization for artificial neural networks: A tabu search application , 1998, Eur. J. Oper. Res..

[28]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[29]  Ralf Salomon,et al.  Evolutionary algorithms and gradient search: similarities and differences , 1998, IEEE Trans. Evol. Comput..

[30]  Antonie Stam,et al.  FOUR APPROACHES TO THE CLASSIFICATION PROBLEM IN DISCRIMINANT ANALYSIS: AN EXPERIMENTAL STUDY* , 1988 .

[31]  Parag C. Pendharkar,et al.  An empirical study of non-binary genetic algorithm-based neural approaches for classification , 1999, ICIS.

[32]  Parag C. Pendharkar,et al.  Nonlinear programming and genetic search application for production scheduling in coal mines , 2000, Ann. Oper. Res..

[33]  Nicola Yanev,et al.  A combinatorial approach to the classification problem , 1999, Eur. J. Oper. Res..

[34]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[35]  Abraham Kandel,et al.  Hybrid Architectures for Intelligent Systems , 1992 .

[36]  Siddhartha Bhattacharyya,et al.  Inductive, Evolutionary, and Neural Computing Techniques for Discrimination: A Comparative Study* , 1998 .

[37]  Bharat A. Jain,et al.  Artificial Neural Network Models for Pricing Initial Public Offerings , 1995 .

[38]  William J. Banks,et al.  An Efficient Optimal Solution Algorithm for the Classification Problem , 1991 .

[39]  Selwyn Piramuthu,et al.  Using Feature Construction to Improve the Performance of Neural Networks , 1998 .