A Lamarckian Approach for Neural Network Training

In Nature, living beings improve their adaptation to surrounding environments by means of two main orthogonal processes: evolution and lifetime learning. Within the Artificial Intelligence arena, both mechanisms inspired the development of non-orthodox problem solving tools, namely: Genetic and Evolutionary Algorithms (GEAs) and Artificial Neural Networks (ANNs). In the past, several gradient-based methods have been developed for ANN training, with considerable success. However, in some situations, these may lead to local minima in the error surface. Under this scenario, the combination of evolution and learning techniques may induce better results, desirably reaching global optima. Comparative tests that were carried out with classification and regression tasks, attest this claim.

[1]  Victor Ciesielski,et al.  An evolutionary approach to training feedforward and recurrent neural networks , 1998, 1998 Second International Conference. Knowledge-Based Intelligent Electronic Systems. Proceedings KES'98 (Cat. No.98EX111).

[2]  Martin A. Riedmiller,et al.  Advanced supervised learning in multi-layer perceptrons — From backpropagation to adaptive learning algorithms , 1994 .

[3]  Lutz Prechelt,et al.  A quantitative study of experimental evaluations of neural network learning algorithms: Current research practice , 1996, Neural Networks.

[4]  Giles Mayley.  The Evolutionary Cost of Learning , 1996 .

[5]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[6]  Christophe G. Giraud-Carrier,et al.  Unifying Learning with Evolution Through Baldwinian Evolution and Lamarckism , 2000, Advances in Computational Intelligence and Learning.

[7]  Maja J. Matarić,et al.  The Evolutionary Cost of Learning , 1996 .

[8]  Larry R. Medsker,et al.  Genetic Algorithms and Neural Networks , 1995 .

[9]  Warren S. Sarle,et al.  Stopped Training and Other Remedies for Overfitting , 1995 .

[10]  J. Ross Quinlan,et al.  Combining Instance-Based and Model-Based Learning , 1993, ICML.

[11]  Abraham Kandel,et al.  Efficient Genetic Algorithms for Training Layered Feedback Neural Networks , 1994, Inf. Sci..

[12]  Miguel Rocha,et al.  Adaptive strategies and the design of evolutionary applications , 1999 .

[13]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[14]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[15]  Paulo Cortez,et al.  Evolving Time Series Forecasting Neural Network Models , 2001 .

[16]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .

[17]  Hiroaki Kitano,et al.  Empirical Studies on the Speed of Convergence of Neural Network Training Using Genetic Algorithms , 1990, AAAI.

[18]  Peter I. Corke,et al.  A robotics toolbox for MATLAB , 1996, IEEE Robotics Autom. Mag..

[19]  Paulo Cortez,et al.  The relationship between learning and evolution in static and dynamic environments , 2000 .

[20]  W. M. Jenkins,et al.  Genetic Algorithms and Neural Networks , 1999, Neural Networks in the Analysis and Design of Structures.

[21]  William Nick Street,et al.  Breast Cancer Diagnosis and Prognosis Via Linear Programming , 1995, Oper. Res..

[22]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[23]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[24]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[25]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[26]  Wan-Chi Siu,et al.  A study of the Lamarckian evolution of recurrent neural networks , 2000, IEEE Trans. Evol. Comput..