SimBa: A novel similarity-based crossover for neuro-evolution

This work presents the SimBa (for Similarity-Based) crossover, a novel crossover operator specifically designed for the evolutionary optimization of neural network topologies that aims at overcoming one of the major problems generally related to the crossover operator, known as the permutation problem. The SimBa crossover starts by looking for a local similarity between two individuals selected from the population. The contribution of each neuron of the layer selected for the crossover is computed, and the neurons of each layer are reordered according to their contribution. Then, each neuron of the layer in the first individual is associated with the most similar neuron of the layer in the other individual, and the neurons of the layer of the second individual are re-ranked by considering the associations with the neurons of the first one. Finally, the neurons above a randomly selected cut-point are swapped to generate the offspring of the selected individuals. An approach exploiting this operator has been implemented and applied to six well-known benchmark classification problems. The experimental results, compared to those obtained by other techniques, show how this new crossover operator can help to produce compact neural networks with satisfactory generalization capability and accuracy.

[1]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[2]  Risto Miikkulainen,et al.  Active Guidance for a Finless Rocket Using Neuroevolution , 2003, GECCO.

[3]  Andrea G. B. Tettamanzi,et al.  Modeling Turning Points in Financial Markets with Soft Computing Techniques , 2010 .

[4]  Ajith Abraham,et al.  Engineering Evolutionary Intelligent Systems , 2008, Studies in Computational Intelligence.

[5]  Kevin N. Gurney,et al.  An introduction to neural networks , 2018 .

[6]  Peter J. B. Hancock,et al.  Genetic algorithms and permutation problems: a comparison of recombination operators for neural net structure specification , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[7]  Md. Monirul Islam,et al.  Graph Matching Recombination for Evolving Neural Networks , 2007, ISNN.

[8]  Martin Mandischer,et al.  Representation and Evolution of Neural Networks , 1993 .

[9]  Antonia Azzini,et al.  Evolving neural networks for static single-position automated trading , 2008 .

[10]  Antonia Azzini,et al.  A neural evolutionary approach to financial modeling , 2006, GECCO '06.

[11]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[12]  Pedro Antonio Gutiérrez,et al.  Evolutionary q-Gaussian radial basis function neural networks for multiclassification , 2011, Neural Networks.

[13]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[14]  Antonia Azzini,et al.  A Neural Evolutionary Classification Method for Brain-Wave Analysis , 2006, EvoWorkshops.

[15]  Heinz Mühlenbein,et al.  The Science of Breeding and Its Application to the Breeder Genetic Algorithm (BGA) , 1993, Evolutionary Computation.

[16]  Marco Castellani,et al.  Evolutionary Artificial Neural Network Design and Training for wood veneer classification , 2009, Eng. Appl. Artif. Intell..

[17]  Nicholas J. Radcliffe,et al.  Genetic set recombination and its application to neural network topology optimisation , 1993, Neural Computing & Applications.

[18]  Stefan Haflidason.  On the significance of the permutation problem in neuroevolution , 2009, GECCO '09.

[19]  P. ZhangG..  Neural networks for classification , 2000 .

[20]  Stefania Marrara,et al.  Dermatology Disease Classification via Novel Evolutionary Artificial Neural Network , 2007 .

[21]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[22]  Pedro Antonio Gutiérrez,et al.  Evolutionary product-unit neural networks classifiers , 2008, Neurocomputing.

[23]  Antonia Azzini,et al.  Evolutionary ANNs: A state of the art survey , 2011, Intelligenza Artificiale.

[24]  Jui-Yu Wu.  MIMO CMAC neural network classifier for solving classification problems , 2011, Appl. Soft Comput..

[25]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[26]  Raymond Chiong,et al.  Novel evolutionary algorithms for supervised classification problems: an experimental study , 2011, Evol. Intell..

[27]  David B. Fogel,et al.  Verifying Anaconda's expert rating by competing against Chinook: experiments in co-evolving a neural checkers player , 2002, Neurocomputing.

[28]  Johan A. K. Suykens,et al.  Genetic Weight Optimization of a Feedforward Neural Network Controller , 1993 .

[29]  David B. Fogel,et al.  The Advantages of Evolutionary Computation , 1997, BCEC.

[30]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[31]  César Hervás-Martínez,et al.  An alternative approach for neural network evolution with a genetic algorithm: Crossover by combinatorial optimization , 2006, Neural Networks.

[32]  César Hervás-Martínez,et al.  COVNET: a cooperative coevolutionary model for evolving artificial neural networks , 2003, IEEE Trans. Neural Networks.

[33]  Mauro Dragoni,et al.  SimBa-2: Improving a novel similarity-based crossover for the evolution of artificial neural networks , 2011, 2011 11th International Conference on Intelligent Systems Design and Applications.

[34]  Mauro Dragoni,et al.  A Novel Similarity-Based Crossover for Artificial Neural Network Evolution , 2010, PPSN.

[35]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[36]  Xin Yao,et al.  Selective negative correlation learning approach to incremental learning , 2009, Neurocomputing.

[37]  David B. Fogel,et al.  Evolutionary program for the identification of dynamical systems , 1997, Defense, Security, and Sensing.

[38]  Arash Bahrammirzaee,et al.  A comparative survey of artificial intelligence applications in finance: artificial neural networks, expert system and hybrid intelligent systems , 2010, Neural Computing and Applications.

[39]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[40]  Pedro Antonio Gutiérrez,et al.  Combined projection and kernel basis functions for classification in evolutionary neural networks , 2009, Neurocomputing.

[41]  Xin Yao,et al.  Recent Advances in Evolutionary Computation , 2006, Journal of Computer Science and Technology.

[42]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[43]  Pedro Antonio Gutiérrez,et al.  Memetic Pareto Evolutionary Artificial Neural Networks to determine growth/no-growth in predictive microbiology , 2011, Appl. Soft Comput..

[44]  F. Ponci,et al.  Incipient Fault Diagnosis in Electrical Drives by Tuned Neural Networks , 2006, 2006 IEEE Instrumentation and Measurement Technology Conference Proceedings.

[45]  Konstantinos P. Ferentinos,et al.  Biological engineering applications of feedforward neural networks designed and parameterized by genetic algorithms , 2005, Neural Networks.

[46]  Antonia Azzini,et al.  A New Genetic Approach for Neural Network Design , 2008, Engineering Evolutionary Intelligent Systems.