Metaheuristic Optimization: Algorithm Analysis and Open Problems

Metaheuristic algorithms are becoming an important part of modern optimization. A wide range of metaheuristic algorithms have emerged over the last two decades, and many metaheuristics such as particle swarm optimization are becoming increasingly popular. Despite their popularity, mathematical analysis of these algorithms lacks behind. Convergence analysis still remains unsolved for the majority of metaheuristic algorithms, while efficiency analysis is equally challenging. In this paper, we intend to provide an overview of convergence and efficiency studies of metaheuristics, and try to provide a framework for analyzing metaheuristics in terms of convergence and efficiency. This can form a basis for analyzing other algorithms. We also outline some open questions as further research topics.

[1]  Frank Neumann,et al.  Bioinspired computation in combinatorial optimization: algorithms and their computational complexity , 2010, GECCO '12.

[2]  Xin-She Yang,et al.  Nature-Inspired Metaheuristic Algorithms , 2008 .

[3]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[4]  Christian Blum,et al.  Metaheuristics in combinatorial optimization: Overview and conceptual comparison , 2003, CSUR.

[5]  P. A. Prince,et al.  Lévy flight search patterns of wandering albatrosses , 1996, Nature.

[6]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[7]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[8]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[9]  Robert L. Smith,et al.  Adaptive search with stochastic acceptance probabilities for global optimization , 2008, Oper. Res. Lett..

[10]  Olivier Teytaud,et al.  Continuous Lunches Are Free Plus the Design of Optimal Optimization Algorithms , 2010, Algorithmica.

[11]  J. Rosenthal,et al.  Markov Chain Monte Carlo , 2018 .

[12]  Peter Rossmanith,et al.  Simulated Annealing , 2008, Taschenbuch der Algorithmen.

[13]  Amitava Chatterjee,et al.  Nonlinear inertia weight variation for dynamic adaptation in particle swarm optimization , 2006, Comput. Oper. Res..

[14]  Xin-She Yang.  Introduction to Mathematical Optimization: From Linear Programming to Metaheuristics , 2008 .

[15]  Joseph A. C. Delaney.  Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.

[16]  Xin-She Yang,et al.  Firefly algorithm, stochastic test functions and design optimisation , 2010, Int. J. Bio Inspired Comput..

[17]  Mirko Krivánek,et al.  Simulated Annealing: A Proof of Convergence , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Chak-Kuen Wong,et al.  Convergence Analysis of Simulated Annealing-Based Algorithms Solving Flow Shop Scheduling Problems , 2000, CIAC.

[19]  Anne Auger,et al.  Theory of Randomized Search Heuristics: Foundations and Recent Developments , 2011, Theory of Randomized Search Heuristics.

[20]  Panos M. Pardalos,et al.  Encyclopedia of Optimization , 2006 .

[21]  A M Reynolds,et al.  The Lévy flight paradigm: random search patterns and mechanisms. , 2009, Ecology.

[22]  Xin-She Yang,et al.  Engineering Optimization: An Introduction with Metaheuristic Applications , 2010 .

[23]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[24]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[25]  W. Gilks.  Markov Chain Monte Carlo , 2005 .

[26]  Ilya M. Sobol,et al.  A Primer for the Monte Carlo Method , 1994 .

[27]  Xin-She Yang,et al.  Engineering optimisation by cuckoo search , 2010, Int. J. Math. Model. Numer. Optimisation.

[28]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[29]  Kathryn A. Dowsland,et al.  Simulated Annealing , 1989, Encyclopedia of GIS.

[30]  Panos M. Pardalos,et al.  Complexity analysis for maximum flow problems with arc reversals , 2010, J. Comb. Optim..

[31]  Marco Dorigo,et al.  Optimization, Learning and Natural Algorithms , 1992 .

[32]  Marek Gutowski.  L\'evy flights as an underlying mechanism for global optimization algorithms , 2001 .

[33]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[34]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[35]  Carsten Witt,et al.  Bioinspired Computation in Combinatorial Optimization , 2010, Bioinspired Computation in Combinatorial Optimization.

[36]  Ilya Pavlyukevich.  Lévy flights, non-local search and simulated annealing , 2007, J. Comput. Phys..

[37]  Panos M. Pardalos,et al.  A Collection of Test Problems for Constrained Global Optimization Algorithms , 1990, Lecture Notes in Computer Science.

[38]  J. L. Nolan.  Stable Distributions. Models for Heavy Tailed Data , 2001 .

[40]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .