On the issue of separability for problem decomposition in cooperative neuro-evolution

Abstract Cooperative coevolution divides an optimisation problem into subcomponents and employs evolutionary algorithms for evolving them. Problem decomposition has been a major issue in using cooperative coevolution for neuro-evolution. Efficient problem decomposition methods group interacting variables into the same subcomponents. It is important to find out which problem decomposition methods efficiently group subcomponents and the behaviour of neural network during training in terms of the interaction among the synapses. In this paper, the interdependencies among the synapses are analysed and a problem decomposition method is introduced for feedforward neural networks on pattern classification problems. We show that the neural network training problem is partially separable and that the level of interdependencies changes during the learning process. The results confirm that the proposed problem decomposition method has improved performance compared to its counterparts.

[1]  N. Garc'ia-Pedrajas,et al.  CIXL2: A Crossover Operator for Evolutionary Algorithms Based on Population Features , 2005, J. Artif. Intell. Res..

[2]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[3]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[4]  César Hervás-Martínez,et al.  Multi-objective cooperative coevolution of artificial neural networks (multi-objective cooperative networks) , 2002, Neural Networks.

[5]  Nicolás García-Pedrajas,et al.  A cooperative constructive method for neural networks for pattern recognition , 2007, Pattern Recognit..

[6]  Risto Miikkulainen,et al.  Accelerated Neural Evolution through Cooperatively Coevolved Synapses , 2008, J. Mach. Learn. Res..

[7]  Hongfei Teng,et al.  Cooperative Co-evolutionary Differential Evolution for Function Optimization , 2005, ICNC.

[8]  Jürgen Schmidhuber,et al.  Training Recurrent Networks by Evolino , 2007, Neural Computation.

[9]  Xin Yao,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008, Inf. Sci..

[10]  X. Yao,et al.  Scaling up fast evolutionary programming with cooperative coevolution , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[11]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[12]  Mengjie Zhang,et al.  Encoding subcomponents in cooperative co-evolutionary recurrent neural networks , 2011, Neurocomputing.

[13]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[14]  Kwong-Sak Leung,et al.  Data mining of Bayesian networks using cooperative coevolution , 2004, Decis. Support Syst..

[15]  César Hervás-Martínez,et al.  COVNET: a cooperative coevolutionary model for evolving artificial neural networks , 2003, IEEE Trans. Neural Networks.

[16]  Kalyanmoy Deb,et al.  A Computationally Efficient Evolutionary Algorithm for Real-Parameter Optimization , 2002, Evolutionary Computation.

[17]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[18]  César Hervás-Martínez,et al.  Cooperative coevolution of artificial neural network ensembles for pattern classification , 2005, IEEE Transactions on Evolutionary Computation.

[19]  Yew-Soon Ong,et al.  Advances in Natural Computation, First International Conference, ICNC 2005, Changsha, China, August 27-29, 2005, Proceedings, Part I , 2005, ICNC.

[20]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[21]  Jiuyong Li.  AI 2010: Advances in Artificial Intelligence - 23rd Australasian Joint Conference, Adelaide, Australia, December 7-10, 2010. Proceedings , 2011, Australasian Conference on Artificial Intelligence.

[22]  Risto Miikkulainen,et al.  Robust non-linear control through neuroevolution , 2003 .

[23]  Mengjie Zhang,et al.  An Encoding Scheme for Cooperative Coevolutionary Feedforward Neural Networks , 2010, Australasian Conference on Artificial Intelligence.