Evaluating the process of a genetic algorithm to improve the back-propagation network: A Monte Carlo study

Many studies have mapped a bit-string genotype using a genetic algorithm to represent network architectures to improve performance of back-propagation networks (BPN). But the limitations of gradient search techniques applied to complex nonlinear optimization problems have often resulted in inconsistent and unpredictable performance. This study focuses on how to collect and re-evaluate the weight matrices of a BPN while the genetic algorithm operations are processing in each generation to optimize the weight matrices. In this way, overfitting, a drawback of BPNs that usually occurs during the later stage of neural network training with descending training error and ascending prediction error, can also be avoided. This study extends the parameters and topology of the neural network to enhance the feasibility of the solution space for complex nonlinear problems. The value of the proposed model is compared with previous studies using a Monte Carlo study on in-sample, interpolation, and extrapolation data for six test functions.

[1]  Luigi Fortuna,et al.  Genetic algorithms to select optimal neural network topology , 1992, [1992] Proceedings of the 35th Midwest Symposium on Circuits and Systems.

[2]  Ru-Qin Yu,et al.  Multivariate nonlinear modelling of fluorescence data by neural network with hidden node pruning algorithm , 1997 .

[3]  Claudio Moraga,et al.  Optimization of feedforward neural networks , 1996 .

[4]  Brian D. Ripley,et al.  Statistical aspects of neural networks , 1993 .

[5]  Lalit M. Patnaik,et al.  Adaptive probabilities of crossover and mutation in genetic algorithms , 1994, IEEE Trans. Syst. Man Cybern..

[6]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[7]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[8]  Gustavo Deco,et al.  Two Strategies to Avoid Overfitting in Feedforward Networks , 1997, Neural Networks.

[9]  Yang Han,et al.  A neural network model based on the multi-stage optimization approach for short-term food price forecasting in China , 2007, Expert Syst. Appl..

[10]  Yimo Zhang,et al.  Inspection of defects in optical fibers based on back-propagation neural networks , 2001 .

[11]  Pei-Chann Chang,et al.  Fuzzy Delphi and back-propagation model for sales forecasting in PCB industry , 2006, Expert Syst. Appl..

[12]  Manel Poch,et al.  Evaluation of natural computation techniques in the modelling and optimization of a sequential injection flow system for colorimetric iron(III) determination , 1997 .

[13]  Randall S. Sexton,et al.  Toward global optimization of neural networks: A comparison of the genetic algorithm and backpropagation , 1998, Decis. Support Syst..

[14]  Y. Ikuno,et al.  Application of an Improved Genetic Algorithm to the Learning of Neural Networks , 1994 .

[15]  Chao-Ton Su,et al.  Applying robust multi-response quality engineering for parameter selection using a novel neural-genetic algorithm , 2003, Comput. Ind..

[16]  Ingoo Han,et al.  Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index , 2000 .

[17]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[18]  B. S. Lim,et al.  Optimal design of neural networks using the Taguchi method , 1995, Neurocomputing.

[19]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[20]  Zhiye Zhao,et al.  Design of structural modular neural networks with genetic algorithm , 2003 .

[21]  Ali K. Kamrani,et al.  A genetic algorithm-based solution methodology for modular design , 2003, J. Intell. Manuf..

[22]  Randall S. Sexton,et al.  Comparing backpropagation with a genetic algorithm for neural network training , 1999 .

[23]  Ben S. Gerber,et al.  Use of genetic algorithms for neural networks to predict community-acquired pneumonia , 2004, Artif. Intell. Medicine.

[24]  Michael Y. Hu,et al.  A simulation study of artificial neural networks for nonlinear time-series forecasting , 2001, Comput. Oper. Res..

[25]  K. R. Al-Balushi,et al.  Artificial neural networks and support vector machines with genetic algorithm for bearing fault detection , 2003 .

[26]  W. Vach,et al.  Neural networks and logistic regression: Part I , 1996 .