Cooperative Co-Evolution With Differential Grouping for Large Scale Optimization

Cooperative co-evolution has been introduced into evolutionary algorithms with the aim of solving increasingly complex optimization problems through a divide-and-conquer paradigm. In theory, the idea of co-adapted subcomponents is desirable for solving large-scale optimization problems. However, in practice, without prior knowledge about the problem, it is not clear how the problem should be decomposed. In this paper, we propose an automatic decomposition strategy called differential grouping that can uncover the underlying interaction structure of the decision variables and form subcomponents such that the interdependence between them is kept to a minimum. We show mathematically how such a decomposition strategy can be derived from a definition of partial separability. The empirical studies show that such near-optimal decomposition can greatly improve the solution quality on large-scale global optimization problems. Finally, we show how such an automated decomposition allows for a better approximation of the contribution of various subcomponents, leading to a more efficient assignment of the computational budget to various subcomponents.

[1]  R. Salomon Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[2]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[3]  Karsten Weicker,et al.  On the improvement of coevolutionary optimizers by learning variable interdependencies , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[4]  Zhenyu Yang,et al.  Large-Scale Global Optimization Using Cooperative Coevolution with Variable Interaction Learning , 2010, PPSN.

[5]  David E. Goldberg,et al.  Linkage Identification by Non-monotonicity Detection for Overlapping Functions , 1999, Evolutionary Computation.

[6]  David E. Goldberg,et al.  Dependency Structure Matrix, Genetic Algorithms, and Effective Recombination , 2009, Evolutionary Computation.

[7]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[8]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[9]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[10]  Xin Yao,et al.  Multilevel cooperative coevolution for large scale optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[11]  Jeffrey Horn,et al.  Handbook of evolutionary computation , 1997 .

[12]  Ryan Bishop,et al.  Of Method , 2007 .

[13]  D. Goldberg,et al.  A Survey of Linkage Learning Techniques in Genetic and Evolutionary Algorithms , 2007 .

[14]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[15]  Xiaodong Li,et al.  Cooperative Co-evolution for large scale optimization through more frequent random grouping , 2010, IEEE Congress on Evolutionary Computation.

[16]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[17]  Hongfei Teng,et al.  Cooperative Co-evolutionary Differential Evolution for Function Optimization , 2005, ICNC.

[18]  M Ptashne,et al.  How gene activators work. , 1989, Scientific American.

[19]  Jim Smith,et al.  An Adaptive Poly-Parental Recombination Strategy , 1995, Evolutionary Computing, AISB Workshop.

[20]  Raymond Chiong,et al.  Evolutionary Optimization: Pitfalls and Booby Traps , 2012, Journal of Computer Science and Technology.

[21]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[22]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[23]  George B. Dantzig,et al.  Decomposition Principle for Linear Programs , 1960 .

[24]  D.E. Goldberg,et al.  A genetic algorithm using linkage identification by nonlinearity check , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[25]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[26]  Francisco Herrera,et al.  MA-SW-Chains: Memetic algorithm based on local search chains for large scale continuous global optimization , 2010, IEEE Congress on Evolutionary Computation.

[27]  P. Toint,et al.  Local convergence analysis for partitioned quasi-Newton updates , 1982 .

[28]  Kalyanmoy Deb,et al.  RapidAccurate Optimization of Difficult Problems Using Fast Messy Genetic Algorithms , 1993, ICGA.

[29]  N. Rescher Discourse on a Method , 1968 .

[30]  X. Yao,et al.  Scaling up fast evolutionary programming with cooperative coevolution , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[31]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[32]  David E. Goldberg,et al.  Combining The Strengths Of Bayesian Optimization Algorithm And Adaptive Evolution Strategies , 2002, GECCO.

[33]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[34]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[35]  Hillol Kargupta,et al.  The performance of the gene expression messy genetic algorithm on real test functions , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[36]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[37]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[38]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[39]  Xin Yao,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008, Inf. Sci..

[40]  G. Harik Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms , 1997 .

[41]  Xiaodong Li,et al.  Smart use of computational resources based on contribution for cooperative co-evolutionary algorithms , 2011, GECCO '11.

[42]  Xiaodong Li,et al.  Cooperative Co-evolution with delta grouping for large scale non-separable function optimization , 2010, IEEE Congress on Evolutionary Computation.

[43]  Xiaodong Li,et al.  Cooperatively Coevolving Particle Swarms for Large Scale Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[44]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[45]  William S. Klug,et al.  Concepts of Genetics , 1999 .

[46]  Yuval Davidor,et al.  Epistasis Variance: Suitability of a Representation to Genetic Algorithms , 1990, Complex Syst..

[47]  Xin Yao,et al.  Self-adaptive differential evolution with neighborhood search , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[48]  YaoXin,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008 .

[49]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[50]  Masaharu Munetomo,et al.  Linkage Identification by Nonlinearity Check for Real-Coded Genetic Algorithms , 2004, GECCO.

[51]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.