A comparative study of different local search application strategies in hybrid metaheuristics

This paper presents results of a comparative study with the objective to identify the most effective and efficient way of applying a local search method embedded in a hybrid algorithm. The hybrid metaheuristic employed in this study is called ''DE-HS-HJ'' because it is comprised of two cooperative metaheusitic algorithms, i.e., differential evolution (DE) and harmony search (HS), and one local search (LS) method, i.e., Hooke and Jeeves (HJ) direct search. Eighteen different ways of using HJ local search were implemented and all of them were evaluated with 19 problems, in terms of six performance indices, covering both accuracy and efficiency. Statistic analyses were conducted accordingly to determine the significance in performance differences. The test results show that overall the best three LS application strategies are applying local search to every generated solution with a specified probability and also to each newly updated solution (NUS+ESP), applying local search to every generated solution with a specified probability (ESP), and applying local search to every generated solution with probability and also to the updated current global best solution (EUGbest+ESP). ESP is found to be the best local search application strategy in terms of success rate. Integrating it with NUS further improve the overall performance. EUGbest+ESP is the most efficient and it is also able to achieve high level of accuracy (the fourth place in terms of success rate with an average above 0.9).

[1]  J. Hsu.  Simultaneous Confidence Intervals for all Distances from the "Best" , 1981 .

[2]  Pandian Vasant,et al.  Hybrid LS-SA-PS methods for solving fuzzy non-linear programming problems , 2013, Math. Comput. Model..

[3]  Pandian Vasant,et al.  Hybrid pattern search and simulated annealing for fuzzy production planning problems , 2010, Comput. Math. Appl..

[4]  Reza Tavakkoli-Moghaddam,et al.  A novel hybrid approach combining electromagnetism-like method with Solis and Wets local search for continuous optimization problems , 2009, J. Glob. Optim..

[5]  Kaisa Miettinen,et al.  Efficient hybrid methods for global continuous optimization based on simulated annealing , 2006, Comput. Oper. Res..

[6]  Ta-Cheng Chen,et al.  IAs based approach for reliability redundancy allocation problems , 2006, Appl. Math. Comput..

[7]  Witold Pedrycz,et al.  Hybrid optimization with improved tabu search , 2011, Applied Soft Computing.

[8]  Patrick Siarry,et al.  Genetic and Nelder-Mead algorithms hybridized for a more accurate global optimization of continuous multiminima functions , 2003, Eur. J. Oper. Res..

[9]  Bo Liu,et al.  Improved particle swarm optimization combined with chaos , 2005 .

[10]  Rong-Song He,et al.  Improving real-parameter genetic algorithm with simulated annealing for engineering problems , 2006, Adv. Eng. Softw..

[11]  Pierre Hansen,et al.  Cooperative Parallel Variable Neighborhood Search for the p-Median , 2004, J. Heuristics.

[12]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[13]  S. Kitayama,et al.  A method for mixed integer programming problems by particle swarm optimization , 2006 .

[14]  G. Thierauf,et al.  A combined heuristic optimization technique , 2005, Adv. Eng. Softw..

[15]  Anikó Csébfalvi,et al.  A hybrid meta-heuristic method for continuous engineering optimization , 2009 .

[16]  James C. Bezdek,et al.  On cluster validity for the fuzzy c-means model , 1995, IEEE Trans. Fuzzy Syst..

[17]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[18]  Günther R. Raidl,et al.  A Unified View on Hybrid Metaheuristics , 2006, Hybrid Metaheuristics.

[19]  Carlos Cruz Corona,et al.  Using memory and fuzzy rules in a co-operative multi-thread strategy for optimization , 2006, Inf. Sci..

[20]  Saïd Salhi,et al.  A hybrid algorithm for identifying global and local minima when optimizing functions with many minima , 2004, Eur. J. Oper. Res..

[21]  T. Warren Liao,et al.  Two hybrid differential evolution algorithms for engineering design optimization , 2010, Appl. Soft Comput..

[22]  Xin Yao,et al.  An Experimental Study of Hybridizing Cultural Algorithms and Local Search , 2008, Int. J. Neural Syst..

[23]  B. Babu,et al.  Optimization of process synthesis and design problems: A modified differential evolution approach , 2006 .

[24]  Yuhui Qiu,et al.  A New Hybrid NM Method and Particle Swarm Algorithm for Multimodal Function Optimization , 2005, IDA.

[25]  Mei Zhao,et al.  A niche hybrid genetic algorithm for global optimization of continuous multimodal functions , 2005, Appl. Math. Comput..

[26]  R. Saravanan,et al.  Selection of optimal machining parameters for multi-tool milling operations using a memetic algorithm , 2006 .

[27]  Pandian Vasant,et al.  HYBRID SIMULATED ANNEALING AND GENETIC ALGORITHMS FOR INDUSTRIAL PRODUCTION MANAGEMENT PROBLEMS , 2009 .

[28]  Masao Fukushima,et al.  Tabu Search directed by direct search methods for nonlinear global optimization , 2006, Eur. J. Oper. Res..

[29]  Henry Braun,et al.  The Collected Works of John W. Tukey: Volume VIII Multiple Comparisons 1948-1983. , 1995 .

[30]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[31]  Masao Fukushima,et al.  Heuristic pattern search and its hybridization with simulated annealing for nonlinear global optimization , 2004, Optim. Methods Softw..

[32]  M. Fesanghary,et al.  An improved harmony search algorithm for solving optimization problems , 2007, Appl. Math. Comput..

[33]  M. Mahdavi,et al.  ARTICLE IN PRESS Available online at www.sciencedirect.com , 2007 .

[34]  O. S. Soliman,et al.  Memetic Particle Swarm Optimization algorithm for multi-objective optimization problems , 2012, 2012 8th International Conference on Informatics and Systems (INFOS).

[35]  Helio J. C. Barbosa,et al.  A GA-Simplex Hybrid Algorithm for Global Minimization of Molecular Potential Energy Functions , 2005, Ann. Oper. Res..

[36]  Pandian Vasant,et al.  Hybrid genetic algorithms and line search method for industrial production planning with non-linear fitness function , 2009, Eng. Appl. Artif. Intell..

[37]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[38]  Mitsuo Gen,et al.  Genetic algorithm for non-linear mixed integer programming problems and its applications , 1996 .

[39]  Feng-Sheng Wang,et al.  Hybrid method of evolutionary algorithms for static and dynamic optimization problems with application to a fed-batch fermentation process , 1999 .

[40]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[41]  Shu-Kai S. Fan,et al.  A genetic algorithm and a particle swarm optimizer hybridized with Nelder-Mead simplex search , 2006, Comput. Ind. Eng..

[42]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[43]  Fatih Erdogan Sevilgen,et al.  A Hybrid Particle Swarm Optimization Algorithm for Function Optimization , 2008, EvoWorkshops.

[44]  A. C. Martínez-Estudillo,et al.  Hybridization of evolutionary algorithms and local search by means of a clustering method , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[45]  Ranjan Ganguli,et al.  An automated hybrid genetic-conjugate gradient algorithm for multimodal optimization problems , 2005, Appl. Math. Comput..

[46]  Zong Woo Geem,et al.  A New Heuristic Optimization Algorithm: Harmony Search , 2001, Simul..

[47]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[48]  T. S. Lee,et al.  A particle swarm approach for grinding process optimization analysis , 2007 .

[49]  V. K. Jayaraman,et al.  Ant Colony Approach to Continuous Function Optimization , 2000 .

[50]  Günther R. Raidi.  A unified view on hybrid metaheuristics , 2006 .

[51]  Thomas Stützle,et al.  Hybrid Particle Swarm Optimization: An Examination of the Influence of Iterative Improvement Algorithms on Performance , 2006, ANTS Workshop.

[52]  K. Deb.  An Efficient Constraint Handling Method for Genetic Algorithms , 2000 .

[53]  M. Carmen Garrido,et al.  Using machine learning in a cooperative hybrid parallel strategy of metaheuristics , 2009, Inf. Sci..

[54]  Linet Özdamar,et al.  Investigating a hybrid simulated annealing and local search algorithm for constrained optimization , 2008, Eur. J. Oper. Res..

[55]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[56]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[57]  El-Ghazali Talbi,et al.  A Taxonomy of Hybrid Metaheuristics , 2002, J. Heuristics.

[58]  A. Ebenezer Jeyakumar,et al.  Hybrid PSO–SQP for economic dispatch with valve-point effect , 2004 .