Neural network construction and training using grammatical evolution

The term neural network evolution usually refers to network topology evolution leaving the network's parameters to be trained using conventional algorithms. In this paper we present a new method for neural network evolution that evolves the network topology along with the network parameters. The proposed method uses grammatical evolution to encode both the network and the parameters space. This allows for a better description of the network using a formal grammar allowing the network architect to shape the resulting search space in order to meet each problem requirement. The proposed method is compared with other three methods for neural network training and is evaluated using 9 known classification problems and 9 known regression problems. In all 18 datasets, the proposed method outperforms its competitors.

[1]  Sung-Bae Cho,et al.  Modular neural networks evolved by genetic programming , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[2]  S.E. Eklund,et al.  Time series forecasting using massively parallel genetic programming , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[3]  Kaisa Miettinen,et al.  Evolutionary algorithms in engineering and computer science : recent advances in genetic algorithms, evolution strategies, evolutionary programming, genetic programming and industrial applications , 1999 .

[4]  A. Yamazaki,et al.  Optimization of neural network weights and architectures for odor recognition using simulated annealing , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[5]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (2nd, extended ed.) , 1994 .

[6]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[7]  Riccardo,et al.  Evolution of the Topology and the Weightsof Neural Networks using GeneticProgramming with a Dual RepresentationJo , 1997 .

[8]  Kwong-Sak Leung,et al.  Data Classification Using Genetic Parallel Programming , 2003, GECCO.

[9]  Michael O'Neill,et al.  Grammatical evolution - evolutionary automatic programming in an arbitrary language , 2003, Genetic programming.

[10]  Ge Yu,et al.  Method of evolutionary neural network-based intrusion detection , 2001, 2001 International Conferences on Info-Tech and Info-Net. Proceedings (Cat. No.01EX479).

[11]  Kwong-Sak Leung,et al.  Evolving data classification programs using genetic parallel programming , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[12]  Conor Ryan,et al.  Grammatical evolution , 2007, GECCO '07.

[13]  Riccardo Poli,et al.  Evolving the Topology and the Weights of Neural Networks Using a Dual Representation , 2004, Applied Intelligence.

[14]  R. Poli,et al.  Discovery of Symbolic, Neuro-Symbolic and Neural Networks with Parallel Distributed Genetic Programming , 1997, ICANNGA.

[15]  M. J. D. Powell,et al.  A tolerant algorithm for linearly constrained optimization calculations , 1989, Math. Program..

[16]  Hussein A. Abbass,et al.  An evolutionary artificial neural networks approach for breast cancer diagnosis , 2002, Artif. Intell. Medicine.

[17]  Sung-Bae Cho,et al.  Evolutionary neural networks for anomaly detection based on the behavior of a program , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Sung-Bae Cho,et al.  Evolved neural networks based on cellular automata for sensory-motor controller , 2006, Neurocomputing.

[19]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[20]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[21]  Chrysostomos D. Stylios,et al.  Novel approach for fetal heart rate classification introducing grammatical evolution , 2007, Biomed. Signal Process. Control..

[22]  田中 博,et al.  Proceedings of the Fifth international symposium on artificial life and robotics (AROB 5th'00) : for human welfare and artificial life robotics : January 26-28, 2000 Compal hall, Oita, Japan , 2000 .

[23]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[24]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[25]  Yu Li,et al.  Particle swarm optimisation for evolving artificial neural network , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[26]  Wei Gao.  Study on new evolutionary neural network , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[27]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[28]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[29]  GavrilisDimitris,et al.  Neural network construction and training using grammatical evolution , 2008 .

[30]  Sung-Bae Cho,et al.  Evolving artificial neural networks for DNA microarray analysis , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[31]  H.S. Lopes,et al.  A parallel genetic algorithm for rule discovery in large databases , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[32]  D. Gavrilis,et al.  Classification of fetal heart rate using grammatical evolution , 2005, IEEE Workshop on Signal Processing Systems Design and Implementation, 2005..

[33]  A AbbassHussein.  An evolutionary artificial neural networks approach for breast cancer diagnosis , 2002 .

[34]  Rudolf F. Albrecht,et al.  Artificial Neural Nets and Genetic Algorithms , 1995, Springer Vienna.

[35]  John R. Koza,et al.  Genetic generation of both the weights and architecture for a neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[36]  Juan Julián Merelo Guervós,et al.  Evolving RBF neural networks for time-series forecasting with EvRBF , 2004, Inf. Sci..

[37]  James A. Reggia,et al.  Evolutionary Design of Neural Network Architectures Using a Descriptive Encoding Language , 2006, IEEE Transactions on Evolutionary Computation.

[38]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[39]  D. Gavrilis,et al.  Introducing Grammatical Evolution in Fetal Heart Rate Analysis and Classification , 2006, 2006 3rd International IEEE Conference Intelligent Systems.

[40]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[41]  Isaac E. Lagaris,et al.  Program Summary , 1986, PMLA/Publications of the Modern Language Association of America.

[42]  Conor Ryan,et al.  Grammatical Evolution , 2001, Genetic Programming Series.