Multi-optima search using Differential Evolution and unsupervised clustering

The aim of this paper is the combination of an Evolutionary Algorithm and a Data Mining technique for the location and computation of multiple local and global optima of an objective function. To accomplish this task we exploit the spatial concentration of the population members around the optima of the objective function. Such concentration regions are determined by applying clustering algorithms on the actual positions of the members of the population. Subsequently, the evolutionary search is confined in the interior of the regions discovered. To enable the simultaneous discovery of more than one global and local optima, we propose the use of unsupervised clustering algorithms that also provide intuitive approximations for the number of clusters. Furthermore, as shown by the experimental analysis, the proposed scheme has often the potential of accelerating the convergence speed of the Evolutionary Algorithm, without the need for extra function evaluations.

[1]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[2]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[3]  Michael N. Vrahatis,et al.  On the alleviation of the problem of local minima in back-propagation , 1997 .

[4]  Bruno Sareni,et al.  Fitness sharing and niching methods revisited , 1998, IEEE Trans. Evol. Comput..

[5]  Vassilis P. Plagianakos,et al.  Parallel evolutionary training algorithms for “hardware-friendly” neural networks , 2002, Natural Computing.

[6]  Susana Gómez,et al.  The tunnelling method for solving the constrained global optimization problem with several non-connected feasible regions , 1982 .

[7]  Kalyanmoy Deb,et al.  Omni-optimizer: A generic evolutionary algorithm for single and multi-objective optimization , 2008, Eur. J. Oper. Res..

[8]  D. Goldberg,et al.  Adaptive Niching via coevolutionary Sharing , 1997 .

[9]  A. A. Torn.  Cluster Analysis Using Seed Points and Density-Determined Hyperspheres as an Aid to Global Optimization , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[10]  Leen Stougie,et al.  Global optimization : a stochastic approach , 1980 .

[11]  Michael J. Shaw,et al.  Genetic algorithms with dynamic niche sharing for multimodal function optimization , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[12]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[13]  Dimitris K. Tasoulis,et al.  Enhancing principal direction divisive clustering , 2010, Pattern Recognit..

[14]  Jie Yao,et al.  On Clustering in Evolutionary Computation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[15]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[16]  Kevin Warwick,et al.  A Genetic Algorithm with Dynamic Niche Clustering for Multimodal Function Optimisation , 1999, ICANNGA.

[17]  Daniel Boley,et al.  Principal Direction Divisive Partitioning , 1998, Data Mining and Knowledge Discovery.

[18]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[19]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[20]  Jouni Lampinen,et al.  A Trigonometric Mutation Operation to Differential Evolution , 2003, J. Glob. Optim..

[21]  Ling Qing,et al.  Crowding clustering genetic algorithm for multimodal function optimization , 2006 .

[22]  P. John Clarkson,et al.  A Species Conserving Genetic Algorithm for Multimodal Function Optimization , 2002, Evolutionary Computation.

[23]  George D. Magoulas,et al.  Evolutionary training of hardware realizable multilayer perceptrons , 2006, Neural Computing & Applications.

[24]  M. N. Vrahatis,et al.  Objective function “stretching” to alleviate convergence to local minima , 2001 .

[25]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[26]  Shigenobu Kobayashi,et al.  Adaptive isolation model using data clustering for multimodal function optimization , 2005, GECCO '05.

[27]  Dimitris K. Tasoulis,et al.  Human Designed Vs. Genetically Programmed Differential Evolution Operators , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[28]  J. F. Price,et al.  On descent from local minima , 1971 .

[29]  J. Kennedy,et al.  Stereotyping: improving particle swarm performance with cluster analysis , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[30]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[31]  Dimitris K. Tasoulis,et al.  Enhancing Differential Evolution Utilizing Proximity-Based Mutation Operators , 2011, IEEE Transactions on Evolutionary Computation.

[32]  A. Engelbrecht,et al.  Using vector operations to identify niches for particle swarm optimization , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[33]  B. M. Shchedrin,et al.  A method of finding the global minimum of a function of one variable , 1975 .

[34]  Andreas Zell,et al.  A Clustering Based Niching EA for Multimodal Search Spaces , 2003, Artificial Evolution.

[35]  Vipin Kumar,et al.  The Challenges of Clustering High Dimensional Data , 2004 .

[36]  Jürgen Schmidhuber,et al.  Self-organizing nets for optimization , 2004, IEEE Transactions on Neural Networks.

[37]  Rainer Storn,et al.  System design by constraint adaptation and differential evolution , 1999, IEEE Trans. Evol. Comput..

[38]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[39]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[40]  C. Chute,et al.  An Overview of Statistical Methods for the Classification and Retrieval of Patient Events , 1995, Methods of Information in Medicine.

[41]  Frans van den Bergh,et al.  A NICHING PARTICLE SWARM OPTIMIZER , 2002 .

[42]  Panos M. Pardalos,et al.  State of the Art in Global Optimization , 1996 .

[43]  P. Pardalos,et al.  State of the art in global optimization: computational methods and applications , 1996 .

[44]  Dimitris K. Tasoulis,et al.  Clustering in evolutionary algorithms to efficiently compute simultaneously local and global minima , 2005, 2005 IEEE Congress on Evolutionary Computation.

[45]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[46]  Dimitris K. Tasoulis,et al.  Parallel differential evolution , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).