Studies on Metaheuristics for Continuous Global Optimization Problems

Preface The interface between computer science and operations research has drawn much attention recently especially in optimization which is a main tool in operations research. In optimization area, the interest on this interface has rapidly increased in the last few years in order to develop nonstandard algorithms that can deal with optimization problems which the standard optimization techniques often fail to deal with. Global optimization problems represent a main category of such problems. Global optimization refers to finding the extreme value of a given nonconvex function in a certain feasible region and such problems are classified in two classes; unconstrained and constrained problems. Solving global optimization problems has made great gain from the interest in the interface between computer science and operations research. In general, the classical optimization techniques have difficulties in dealing with global optimization problems. One of the main reasons of their failure is that they can easily be entrapped in local minima. Moreover, these techniques cannot generate or even use the global information needed to find the global minimum for a function with multiple local minima. The interaction between computer science and optimization has yielded new practical solvers for global optimization problems, called metaheuristics. The structures of metaheuristics are mainly based on simulating nature and artificial intelligence tools. Metaheuristics mainly invoke exploration and exploitation search procedures in order to diversify the search all over the search space and intensify the search in some promising areas. Therefore, metaheuristics cannot easily be entrapped in local minima. However, metaheuristics are computationally costly due to their slow convergence. One of the main reasons for their slow convergence is that they may fail to detect promising search directions especially in the vicinity of local minima due to their random constructions. In this study, both global optimization problem classes; unconstrained and constrained problems, are considered in the continuous search space. New hybrid versions of metaheuris-tics are proposed as promising solvers for the considered problems. The proposed methods ii Preface aim to overcome the drawbacks of slow convergence and random constructions of meta-heuristics. In these hybrid methods, local search strategies are inlaid inside metaheuristics in order to guide them especially in the vicinity of local minima, and overcome their slow convergence especially in the final stage of the search. Metaheuristics are derivative-free methods so that direct search methods, which are also derivative-free methods, are invoked to play the role of local search in the …

[1]  M. F. Cardoso,et al.  The simplex-simulated annealing approach to continuous non-linear optimization , 1996 .

[2]  Rafael Martí,et al.  Experimental Testing of Advanced Scatter Search Designs for Global Optimization of Multimodal Functions , 2005, J. Glob. Optim..

[3]  Zbigniew Michalewicz,et al.  Evolutionary Computation 1 , 2018 .

[4]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[5]  F. Schoen TWO-PHASE METHODS FOR GLOBAL OPTIMIZATION , 2002 .

[6]  Masao Fukushima,et al.  Heuristic pattern search and its hybridization with simulated annealing for nonlinear global optimization , 2004, Optim. Methods Softw..

[7]  Emile H. L. Aarts,et al.  Selected Topics in Simulated Annealing , 2002 .

[8]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[9]  Roberto Battiti,et al.  The continuous reactive tabu search: Blending combinatorial optimization and stochastic search for global optimization , 1996, Ann. Oper. Res..

[10]  Tao Wang,et al.  Tuning Strategies in Constrained Simulated Annealing for Nonlinear Global Optimization , 2000, Int. J. Artif. Intell. Tools.

[11]  Fred W. Glover,et al.  A user's guide to tabu search , 1993, Ann. Oper. Res..

[12]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[13]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[14]  Paul Tseng,et al.  Fortified-Descent Simplicial Search Method: A General Approach , 1999, SIAM J. Optim..

[15]  Rafael Martí Multi-Start Methods , 2003, Handbook of Metaheuristics.

[16]  Patrick Siarry,et al.  A Continuous Genetic Algorithm Designed for the Global Optimization of Multimodal Functions , 2000, J. Heuristics.

[17]  Margaret H. Wright,et al.  Direct search methods: Once scorned, now respectable , 1996 .

[18]  Carlos A. Coello Coello,et al.  A simple multimembered evolution strategy to solve constrained optimization problems , 2005, IEEE Transactions on Evolutionary Computation.

[19]  Xin Yao,et al.  Stochastic ranking for constrained evolutionary optimization , 2000, IEEE Trans. Evol. Comput..

[20]  Fred W. Glover,et al.  Tabu Search - Part I , 1989, INFORMS J. Comput..

[21]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[22]  Patrick Siarry,et al.  Tabu Search applied to global optimization , 2000, Eur. J. Oper. Res..

[23]  C. T. Kelley,et al.  Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition , 1999, SIAM J. Optim..

[24]  Juraj Bartolic,et al.  Optimization synthesis of broadband circularly polarized microstrip antennas by hybrid genetic algorithm , 2001 .

[25]  John Yen,et al.  A hybrid approach to modeling metabolic systems using a genetic algorithm and simplex method , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[26]  Charles Audet,et al.  A Pattern Search Filter Method for Nonlinear Programming without Derivatives , 2001, SIAM J. Optim..

[27]  Rafael Martí,et al.  Scatter Search: Diseño Básico y Estrategias avanzadas , 2002, Inteligencia Artif..

[28]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 : Advanced Algorithms and Operators , 2000 .

[29]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[30]  John E. Dennis,et al.  Multidirectional search: a direct search algorithm for parallel machines , 1989 .

[31]  Horst Reiner,et al.  Introduction to Global Optimization. Second Edition , 2000 .

[32]  Jean-Michel Renders,et al.  Hybridizing genetic algorithms with hill-climbing methods for global optimization: two possible ways , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[33]  Thomas Bäck,et al.  Extended Selection Mechanisms in Genetic Algorithms , 1991, ICGA.

[34]  Klaus Schittkowski,et al.  More test examples for nonlinear programming codes , 1981 .

[35]  Yixin Chen,et al.  OPTIMAL ANYTIME SEARCH FOR CONSTRAINED NONLINEAR PROGRAMMING , 2001 .

[36]  Vladimír Kvasnička,et al.  A hybrid of simplex method and simulated annealing , 1997 .

[37]  Masao Fukushima,et al.  Simplex Coding Genetic Algorithm for the Global Optimization of Nonlinear Functions , 2003 .

[38]  Carlos Artemio Coello-Coello,et al.  Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art , 2002 .

[39]  V. J. Torczoit,et al.  Multidirectional search: a direct search algorithm for parallel machines , 1989 .

[40]  P.J.M. van Laarhoven,et al.  Theoretical and Computational Aspects of Simulated Annealing. , 1990 .

[41]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[42]  Der-San Chen,et al.  Continuous optimization by a variant of simulated annealing , 1996, Comput. Optim. Appl..

[43]  M. F. Cardoso,et al.  A simulated annealing approach to the solution of minlp problems , 1997 .

[44]  J. Dennis,et al.  Direct Search Methods on Parallel Machines , 1991 .

[45]  R. Martí,et al.  Métodos Multi-arranque , 2004 .

[46]  F. Franze,et al.  A tabu‐search‐based algorithm for continuous multiminima problems , 2001 .

[47]  P. Pardalos,et al.  Recent developments and trends in global optimization , 2000 .

[48]  Francisco Herrera,et al.  Tackling Real-Coded Genetic Algorithms: Operators and Tools for Behavioural Analysis , 1998, Artificial Intelligence Review.

[49]  Tayfun Gnel A hybrid approach to the synthesis of nonuniform lossy transmission‐line impedance‐matching sections , 2000 .

[50]  Pablo Moscato,et al.  Handbook of Applied Optimization , 2000 .

[51]  I H Osman,et al.  Meta-Heuristics Theory and Applications , 2011 .

[52]  K. Deb An Efficient Constraint Handling Method for Genetic Algorithms , 2000 .

[53]  Patrick Siarry,et al.  Enhanced simulated annealing for globally minimizing functions of many-continuous variables , 1997, TOMS.

[54]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[55]  I. Douglas,et al.  Simple Genetic Algorithm with Local Tuning: Efficient Global Optimizing Technique , 1998 .

[56]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[57]  M. J. D. Powell,et al.  Direct search algorithms for optimization calculations , 1998, Acta Numerica.

[58]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms, Homomorphous Mappings, and Constrained Parameter Optimization , 1999, Evolutionary Computation.

[59]  P. Siarry,et al.  A genetic algorithm with real-value coding to optimize multimodal continuous functions , 2001 .

[60]  Masao Fukushima,et al.  Derivative-Free Filter Simulated Annealing Method for Constrained Continuous Global Optimization , 2006, J. Glob. Optim..

[61]  Z. Michalewicz,et al.  Genocop III: a co-evolutionary algorithm for numerical optimization problems with nonlinear constraints , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[62]  M. E. Johnson,et al.  Generalized simulated annealing for function optimization , 1986 .

[63]  D Cvijovicacute,et al.  Taboo search: an approach to the multiple minima problem. , 1995, Science.

[64]  James E. Baker,et al.  Adaptive Selection Methods for Genetic Algorithms , 1985, International Conference on Genetic Algorithms.

[65]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[66]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[67]  Klaus Schittkowski,et al.  Test examples for nonlinear programming codes , 1980 .

[68]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[69]  Panos M. Pardalos,et al.  Introduction to Global Optimization , 2000, Introduction to Global Optimization.

[70]  K. I. M. McKinnon,et al.  Convergence of the Nelder-Mead Simplex Method to a Nonstationary Point , 1998, SIAM J. Optim..

[71]  Charles Audet,et al.  Analysis of Generalized Pattern Searches , 2000, SIAM J. Optim..

[72]  Sheldon Howard Jacobson,et al.  The Theory and Practice of Simulated Annealing , 2003, Handbook of Metaheuristics.

[73]  N. Hu Tabu search method with random moves for globally optimal design , 1992 .

[74]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[75]  N. R. Chapman,et al.  A hybrid simplex genetic algorithm for estimating geoacoustic parameters using matched-field inversion , 1999 .

[76]  K. Al-Sultan,et al.  A tabu search Hooke and Jeeves algorithm for unconstrained optimization , 1997 .

[77]  C. Ribeiro,et al.  Essays and Surveys in Metaheuristics , 2002, Operations Research/Computer Science Interfaces Series.

[78]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[79]  Rafael Martí,et al.  Intensification and diversification with elite tabu search solutions for the linear ordering problem , 1999, Comput. Oper. Res..

[80]  Carlos F. Torres,et al.  Global Optimization of Gas Allocation to a Group of Wells in Artificial Lift Using Nonlinear Constrained Programming , 2002 .

[81]  Yixin Chen,et al.  Optimal Anytime Constrained Simulated Annealing for Constrained Global Optimization , 2000, CP.

[82]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[83]  David B. Fogel,et al.  Evolution-ary Computation 1: Basic Algorithms and Operators , 2000 .

[84]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 , 2000 .

[85]  M. Powell A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[86]  Masao Fukushima,et al.  Hybrid simulated annealing and direct search method for nonlinear unconstrained global optimization , 2002, Optim. Methods Softw..

[87]  Charles Audet,et al.  Generalized pattern searches with derivative information , 2002, Math. Program..

[88]  Marc Schoenauer,et al.  ASCHEA: new results using adaptive segregational constraint handling , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[89]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[90]  Carlos A. Coello Coello,et al.  Constraint-handling in genetic algorithms through the use of dominance-based tournament selection , 2002, Adv. Eng. Informatics.

[91]  F. Glover,et al.  Handbook of Metaheuristics , 2019, International Series in Operations Research & Management Science.

[92]  William H. Press,et al.  Simulated Annealing Optimization over Continuous Spaces , 1991 .

[93]  M. Fukushima,et al.  Minimizing multimodal functions by simplex coding genetic algorithm , 2003 .

[94]  Masao Fukushima,et al.  Tabu Search directed by direct search methods for nonlinear global optimization , 2006, Eur. J. Oper. Res..

[95]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms for Constrained Parameter Optimization Problems , 1996, Evolutionary Computation.