Hybrid metaheuristics in combinatorial optimization: A survey

Research in metaheuristics for combinatorial optimization problems has lately experienced a noteworthy shift towards the hybridization of metaheuristics with other techniques for optimization. At the same time, the focus of research has changed from being rather algorithm-oriented to being more problem-oriented. Nowadays the focus is on solving the problem at hand in the best way possible, rather than promoting a certain metaheuristic. This has led to an enormously fruitful cross-fertilization of different areas of optimization. This cross-fertilization is documented by a multitude of powerful hybrid algorithms that were obtained by combining components from several different optimization techniques. Hereby, hybridization is not restricted to the combination of different metaheuristics but includes, for example, the combination of exact algorithms and metaheuristics. In this work we provide a survey of some of the most important lines of hybridization. The literature review is accompanied by the presentation of illustrative examples.

[1]  Carlos Cotta dash,et al.  A study of hybridisation techniques and their application to the design of evolutionary algorithms , 1998 .

[2]  W. D. Harvey,et al.  Nonsystematic backtracking search , 1995 .

[3]  Christian Blum,et al.  On the use of different types of knowledge in metaheuristics based on constructing solutions , 2010, Eng. Appl. Artif. Intell..

[4]  Günther R. Raidl,et al.  Multilevel Variable Neighborhood Search for Periodic Routing Problems , 2010, EvoCOP.

[5]  Manuel López-Ibáñez,et al.  Beam-ACO for the travelling salesman problem with time windows , 2010, Comput. Oper. Res..

[6]  Fred W. Glover.  Inequalities and Target Objectives for Metaheuristic Search - Part I: Mixed Binary Optimization , 2008, Advances in Metaheuristics for Hard Optimization.

[7]  Jin-Kao Hao,et al.  A hybrid approach for the 0-1 multidimensional knapsack problem , 2001, IJCAI 2001.

[8]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988 .

[9]  David Pisinger,et al.  An Adaptive Large Neighborhood Search Heuristic for the Pickup and Delivery Problem with Time Windows , 2006, Transp. Sci..

[10]  Thomas Stützle,et al.  Very Large-Scale Neighborhood Search: Overview and Case Studies on Coloring Problems , 2008, Hybrid Metaheuristics.

[11]  Mauro Birattari,et al.  Tuning Metaheuristics - A Machine Learning Perspective , 2009, Studies in Computational Intelligence.

[12]  Celso C. Ribeiro,et al.  Greedy Randomized Adaptive Search Procedures , 2003, Handbook of Metaheuristics.

[13]  Bernd Meyer,et al.  Hybrids of Constructive Metaheuristics and Constraint Programming: A Case Study with ACO , 2008, Hybrid Metaheuristics.

[14]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[15]  C. Reeves.  Modern heuristic techniques for combinatorial problems , 1993 .

[16]  Günther R. Raidl,et al.  A Unified View on Hybrid Metaheuristics , 2006, Hybrid Metaheuristics.

[17]  Saïd Hanafi,et al.  New convergent heuristics for 0-1 mixed integer programming , 2009, Eur. J. Oper. Res..

[18]  Itsuo Hatono,et al.  An Approximate Solution Method for Combinatorial Optimization , 1994 .

[19]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[20]  Enrique Alba,et al.  Optimization Techniques for Solving Complex Problems , 2009 .

[21]  Xavier Gandibleux,et al.  Hybrid Metaheuristics for Multi-objective Combinatorial Optimization , 2008, Hybrid Metaheuristics.

[22]  Alessandro Zanarini,et al.  Deriving Information from Sampling and Diving , 2009, AI*IA.

[23]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[24]  Günther R. Raidl,et al.  The Core Concept for the Multidimensional Knapsack Problem , 2006, EvoCOP.

[25]  Christian Blum,et al.  Hybrid Metaheuristics , 2010, Artificial Intelligence: Foundations, Theory, and Algorithms.

[26]  Leyuan Shi,et al.  An Optimization Framework for Product Design , 2001, Manag. Sci..

[27]  Paul Shaw,et al.  Using Constraint Programming and Local Search Methods to Solve Vehicle Routing Problems , 1998, CP.

[28]  Stefan Voß,et al.  A Math-Heuristic Algorithm for the DNA Sequencing Problem , 2010, LION.

[29]  A. Fréville,et al.  Tabu search: global intensification using dynamic programming , 2006 .

[30]  Andrea Lodi,et al.  Local Search and Constraint Programming , 2003, Handbook of Metaheuristics.

[31]  Chris Walshaw,et al.  A Multilevel Approach to the Travelling Salesman Problem , 2002, Oper. Res..

[32]  Keiichiro Yasuda,et al.  Proximate Optimality Principle Based Tabu Search , 2004 .

[33]  Francisco Barahona,et al.  The volume algorithm: producing primal solutions with a subgradient method , 2000, Math. Program..

[34]  David S. Johnson,et al.  A theoretician's guide to the experimental analysis of algorithms , 1999, Data Structures, Near Neighbor Searches, and Methodology.

[35]  Steven David Prestwich,et al.  The Relation Between Complete and Incomplete Search , 2008, Hybrid Metaheuristics.

[36]  William J. Cook,et al.  The Traveling Salesman Problem: A Computational Study (Princeton Series in Applied Mathematics) , 2007 .

[37]  Vittorio Maniezzo,et al.  Benders decomposition, Lagrangean relaxation and metaheuristic design , 2009, J. Heuristics.

[38]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[39]  Narendra Jussien,et al.  Local search with constraint propagation and conflict-based heuristics , 2000, Artif. Intell..

[40]  Günther R. Raidl,et al.  Combining Lagrangian Decomposition with an Evolutionary Algorithm for the Knapsack Constrained Maximum Spanning Tree Problem , 2007, EvoCOP.

[41]  Chris Walshaw,et al.  Multilevel Refinement for Combinatorial Optimisation Problems , 2004, Ann. Oper. Res..

[42]  Yu-Wang Chen,et al.  Hybrid evolutionary algorithm with marriage of genetic algorithm and extremal optimization for production scheduling , 2008 .

[43]  Christian Blum,et al.  Hybrid Metaheuristics, An Emerging Approach to Optimization , 2008, Hybrid Metaheuristics.

[44]  Roberto Rossi,et al.  Evolving Parameterised Policies for Stochastic Constraint Programming , 2009, CP.

[45]  Marco Dorigo.  Ant colony optimization , 2004, Scholarpedia.

[46]  Enrique Alba,et al.  Parallel Metaheuristics: A New Class of Algorithms , 2005 .

[47]  Vittorio Maniezzo,et al.  VERY STRONGLY CONSTRAINED PROBLEMS: AN ANT COLONY OPTIMIZATION APPROACH , 2008, Cybern. Syst..

[48]  G. Raidl,et al.  An improved genetic algorithm for the multiconstrained 0-1 knapsack problem , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[49]  Sameh Al-Shihabi,et al.  A hybrid of Nested Partition, Binary Ant System, and Linear Programming for the multidimensional knapsack problem , 2010, Comput. Oper. Res..

[50]  Marc Reimann,et al.  Guiding ACO by Problem Relaxation: A Case Study on the Symmetric TSP , 2007, Hybrid Metaheuristics.

[51]  Graham Kendall,et al.  Hyper-Heuristics: An Emerging Direction in Modern Search Technology , 2003, Handbook of Metaheuristics.

[52]  G. Nemhauser,et al.  Integer Programming , 2020 .

[53]  A. L. Soyster,et al.  Zero-one programming with many variables and few constraints , 1978 .

[54]  Carlos Cotta,et al.  A Probabilistic Beam Search Approach to the Shortest Common Supersequence Problem , 2007, EvoCOP.

[55]  Christian Blum,et al.  Beam-ACO - hybridizing ant colony optimization with beam search: an application to open shop scheduling , 2005, Comput. Oper. Res..

[56]  Sunderesh S. Heragu,et al.  A combined branch-and-bound and genetic algorithm based approach for a flowshop scheduling problem , 1996, Ann. Oper. Res..

[57]  Pablo Moscato,et al.  Memetic algorithms: a short introduction , 1999 .

[58]  Kwong-Sak Leung,et al.  A Memetic Algorithm for Multiple-Drug Cancer Chemotherapy Schedule Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[59]  Edward Rothberg,et al.  An Evolutionary Algorithm for Polishing Mixed Integer Programming Solutions , 2007, INFORMS J. Comput..

[60]  S. Imahori,et al.  Improved local search algorithms for the rectangle packing problem with general spatial costs , 2004, Eur. J. Oper. Res..

[61]  Abraham P. Punnen,et al.  Variable Intensity Local Search , 2010, Matheuristics.

[62]  Günther R. Raidl,et al.  An improved hybrid genetic algorithm for the generalized assignment problem , 2004, SAC '04.

[63]  Laurent Perron,et al.  Propagation Guided Large Neighborhood Search , 2004, CP.

[64]  Claude Le Pape,et al.  Exploring relaxation induced neighborhoods to improve MIP solutions , 2005, Math. Program..

[65]  John E. Beasley,et al.  Lagrangian relaxation , 1993 .

[66]  Michel Gendreau,et al.  A Constraint Programming Framework for Local Search Methods , 1999, J. Heuristics.

[67]  Thomas Stützle,et al.  Local search algorithms for combinatorial problems: analysis, algorithms, and new applications , 1999 .

[68]  A. Percus,et al.  Nature's Way of Optimizing , 1999, Artif. Intell..

[69]  Michel Gendreau,et al.  A View of Local Search in Constraint Programming , 1996, CP.

[70]  Enrique Alba,et al.  Parallel Hybrid Metaheuristics , 2005 .

[71]  John E. Beasley,et al.  A Genetic Algorithm for the Multidimensional Knapsack Problem , 1998, J. Heuristics.

[72]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[73]  Stefan Voß,et al.  A Corridor Method-Based Algorithm for the Pre-marshalling Problem , 2009, EvoWorkshops.

[74]  Shun-Feng Su,et al.  Multiple sequence alignment using modified dynamic programming and particle swarm optimization , 2008 .

[75]  Markus Leitner,et al.  Lagrangian Decomposition, Metaheuristics, and Hybrid Approaches for the Design of the Last Mile in Fiber Optic Networks , 2008, Hybrid Metaheuristics.

[76]  Christian Blum,et al.  Beam-ACO for Simple Assembly Line Balancing , 2008, INFORMS J. Comput..

[77]  Patrick Albert,et al.  Strong Combination of Ant Colony Optimization with Constraint Programming Optimization , 2010, CPAIOR.

[78]  Thomas Stützle,et al.  Iterated local search for the quadratic assignment problem , 2006, Eur. J. Oper. Res..

[79]  Patrick Albert,et al.  Integration of ACO in a Constraint Programming Language , 2008, ANTS Conference.

[80]  Carlos García-Martínez,et al.  Hybrid metaheuristics with evolutionary algorithms specializing in intensification and diversification: Overview and progress report , 2010, Comput. Oper. Res..

[81]  Catherine C. McGeoch.  Feature Article - Toward an Experimental Method for Algorithm Simulation , 1996, INFORMS J. Comput..

[82]  Matthew L. Ginsberg,et al.  Limited Discrepancy Search , 1995, IJCAI.

[83]  Temel Öncan,et al.  VLSN search algorithms for partitioning problems using matching neighbourhoods , 2008, J. Oper. Res. Soc..

[84]  Roberto Montemanni,et al.  Heuristic manipulation, tabu search and frequency assignment , 2010, Comput. Oper. Res..

[85]  Claude Lemaréchal,et al.  Lagrangian Relaxation , 2000, Computational Combinatorial Optimization.

[86]  Günter Radons,et al.  Combining evolutionary computation and dynamic programming for solving a dynamic facility layout problem , 2005, Eur. J. Oper. Res..

[87]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[88]  Stefan Voß,et al.  The corridor method: a dynamic programming inspired metaheuristic , 2006 .

[89]  Luiz Antonio Nogueira Lorena,et al.  Clustering Search Heuristic for the Capacitated p -Median Problem , 2008, Innovations in Hybrid Intelligent Systems.

[90]  M. J. Blesa,et al.  Solving the KCT Problem: Large‐Scale Neighborhood Search and Solution Merging , 2009 .

[91]  Joaquín Bautista,et al.  A dynamic programming based heuristic for the assembly line balancing problem , 2009, Eur. J. Oper. Res..

[92]  V. T. Chow,et al.  Discrete Differential Dynamic Programing Approach to Water Resources Systems Optimization , 1971 .

[93]  Helena Ramalhinho Dias Lourenço,et al.  Iterated Local Search , 2001, Handbook of Metaheuristics.

[94]  Hakan Yildiz,et al.  A Large Neighborhood Search Heuristic for Graph Coloring , 2007, CPAIOR.

[95]  Carlos Cotta,et al.  On the Hybridization of Memetic Algorithms With Branch-and-Bound Techniques , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[96]  Leyuan Shi,et al.  Nested Partitions Method for Global Optimization , 2000, Oper. Res..

[97]  Zbigniew Michalewicz,et al.  Feature cluster on adaptation of discrete metaheuristics to continuous optimization , 2008, Eur. J. Oper. Res..

[98]  Chris Walshaw,et al.  Multilevel Refinement for Combinatorial Optimisation: Boosting Metaheuristic Performance , 2008, Hybrid Metaheuristics.

[99]  Chris Walshaw,et al.  Mesh Partitioning: A Multilevel Balancing and Refinement Algorithm , 2000, SIAM J. Sci. Comput..

[100]  Evripidis Bampis,et al.  A multi-start dynasearch algorithm for the time dependent single-machine total weighted tardiness scheduling problem , 2005, Eur. J. Oper. Res..

[101]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Vol. II , 1976 .

[102]  William J. Cook,et al.  The Traveling Salesman Problem: A Computational Study , 2007 .

[103]  Micael Gallego,et al.  GRASP and path relinking for the max-min diversity problem , 2010, Comput. Oper. Res..

[104]  Catherine C. McGeoch.  Experimental analysis of algorithms , 1986 .

[105]  James Smith,et al.  A tutorial for competent memetic algorithms: model, taxonomy, and design issues , 2005, IEEE Transactions on Evolutionary Computation.

[106]  John N. Tsitsiklis,et al.  Rollout Algorithms for Combinatorial Optimization , 1997, J. Heuristics.

[107]  Michel Vasquez,et al.  Improved results on the 0-1 multidimensional knapsack problem , 2005, Eur. J. Oper. Res..

[108]  V. Jeet,et al.  Lagrangian relaxation guided problem space search heuristics for generalized assignment problems , 2007, Eur. J. Oper. Res..

[109]  Günther R. Raidi.  A unified view on hybrid metaheuristics , 2006 .

[110]  Carlos Cotta,et al.  Hybridizations of Metaheuristics With Branch & Bound Derivates , 2008, Hybrid Metaheuristics.

[111]  Roberto Tadei,et al.  An enhanced dynasearch neighborhood for the single-machine total weighted tardiness scheduling problem , 2004, Oper. Res. Lett..

[112]  Andrea Schaerf,et al.  Combining Local Search and Look-Ahead for Scheduling and Constraint Satisfaction Problems , 1997, IJCAI.

[113]  Peter Greistorfer,et al.  A Tabu Scatter Search Metaheuristic for the Arc Routing Problem , 2002 .

[114]  David Pisinger,et al.  Core Problems in Knapsack Algorithms , 1999, Oper. Res..

[115]  James Kennedy,et al.  Proceedings of the 1998 IEEE International Conference on Evolutionary Computation [Book Review] , 1999, IEEE Transactions on Evolutionary Computation.

[116]  Christian Blum,et al.  Metaheuristics in combinatorial optimization: Overview and conceptual comparison , 2003, CSUR.

[117]  S. Binato,et al.  A GRASP FOR JOB SHOP SCHEDULING , 2001 .

[118]  Carlos Cotta,et al.  Embedding Branch and Bound within Evolutionary Algorithms , 2003, Applied Intelligence.

[119]  Chun Hung Cheng,et al.  A hybrid genetic algorithm for the dynamic plant layout problem , 2003 .

[120]  P. Ow,et al.  Filtered beam search in scheduling , 1988 .

[121]  Chris N. Potts,et al.  An Iterated Dynasearch Algorithm for the Single-Machine Total Weighted Tardiness Scheduling Problem , 2002, INFORMS J. Comput..

[122]  Christine Solnon,et al.  Ant Colony Optimization and Constraint Programming , 2010 .

[123]  Carlos García-Martínez,et al.  Memetic Algorithms for Continuous Optimisation Based on Local Search Chains , 2010, Evolutionary Computation.

[124]  Jin-Kao Hao,et al.  A Hybrid Approach for the 01 Multidimensional Knapsack problem , 2001, IJCAI.

[125]  Chun Lu,et al.  An improved GA and a novel PSO-GA-based hybrid algorithm , 2005, Inf. Process. Lett..

[126]  William J. Cook,et al.  A Computational Study of the Job-Shop Scheduling Problem , 1991, INFORMS Journal on Computing.

[127]  Mark Dras,et al.  Kernelization as Heuristic Structure for the Vertex Cover Problem , 2006, ANTS Workshop.

[128]  Pierre Hansen,et al.  Variable neighborhood search and local branching , 2004, Comput. Oper. Res..

[129]  Hisham M. Haddad,et al.  Proceedings of the 2003 ACM Symposium on Applied Computing (SAC), March 9-12, 2003, Melbourne, FL, USA , 2003, SAC.

[130]  Günther R. Raidl,et al.  Variable Neighborhood Search Coupled with ILP-Based Very Large Neighborhood Searches for the (Periodic) Location-Routing Problem , 2010, Hybrid Metaheuristics.

[131]  Mauro Dell'Amico,et al.  On the Integration of Metaheuristic Strategies in Constraint Programming , 2005 .

[132]  Vincent Furnon,et al.  Improved Local Search for CP Toolkits , 2002, Ann. Oper. Res..

[133]  S.J.J. Smith,et al.  Empirical Methods for Artificial Intelligence , 1995 .

[134]  Fred Glover,et al.  Improved Constructive Multistart Strategies for the Quadratic Assignment Problem Using Adaptive Memory , 1999, INFORMS J. Comput..

[135]  John N. Hooker,et al.  Testing heuristics: We have it all wrong , 1995, J. Heuristics.

[136]  Saïd Hanafi,et al.  Improved convergent heuristics for the 0-1 multidimensional knapsack problem , 2011, Ann. Oper. Res..

[137]  Mohamed Haouari,et al.  A hybrid Lagrangian genetic algorithm for the prize collecting Steiner tree problem , 2006, Comput. Oper. Res..

[138]  Steven David Prestwich,et al.  Combining the Scalability of Local Search with the Pruning Techniques of Systematic Search , 2002, Ann. Oper. Res..

[139]  Zbigniew Michalewicz,et al.  Advances in Metaheuristics for Hard Optimization (Natural Computing Series) , 2007 .

[140]  Abraham P. Punnen,et al.  A survey of very large-scale neighborhood search techniques , 2002, Discret. Appl. Math..

[141]  Christian Blum,et al.  Ant colony optimization: Introduction and recent trends , 2005 .

[142]  Matteo Fischetti,et al.  A new ILP-based refinement heuristic for Vehicle Routing Problems , 2006, Math. Program..

[143]  Matthew L. Ginsberg,et al.  Dynamic Backtracking , 1993, J. Artif. Intell. Res..

[144]  Thomas Bartz-Beielstein,et al.  Experimental Methods for the Analysis of Optimization Algorithms , 2010 .

[145]  Günther R. Raidl,et al.  Effective Neighborhood Structures for the Generalized Traveling Salesman Problem , 2008, EvoCOP.

[146]  El-Ghazali Talbi,et al.  ParadisEO: A Framework for the Reusable Design of Parallel and Distributed Metaheuristics , 2004, J. Heuristics.

[147]  J. Christopher Beck.  Solution-Guided Multi-Point Constructive Search for Job Shop Scheduling , 2007, J. Artif. Intell. Res..

[148]  Christian Blum,et al.  A Brief Survey on Hybrid Metaheuristics , 2010 .

[149]  Anton V. Eremeev,et al.  On Complexity of Optimal Recombination for Binary Representations of Solutions , 2008, Evolutionary Computation.

[150]  Vittorio Maniezzo,et al.  Decomposition Techniques as Metaheuristic Frameworks , 2010, Matheuristics.

[151]  Nenad Mladenovic,et al.  Variable neighbourhood decomposition search for 0-1 mixed integer programs , 2009, Comput. Oper. Res..

[152]  R. Bellman.  Dynamic programming. , 1957, Science.

[153]  Stefan Voß,et al.  Applying the corridor method to a blocks relocation problem , 2011, OR Spectr..

[154]  Michel Toulouse,et al.  A multilevel tabu search algorithm for the feature selection problem in biomedical data , 2008, Comput. Math. Appl..

[155]  Thomas Stützle,et al.  Combinations of Local Search and Exact Algorithms , 2003, EvoWorkshops.

[156]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[157]  Antonella Carbonaro,et al.  An ANTS heuristic for the frequency assignment problem , 2000, Future Gener. Comput. Syst..

[158]  Andrea Lodi,et al.  CP-Based Local Branching , 2007, CP.

[159]  J. Christopher Beck,et al.  A Hybrid Constraint Programming / Local Search Approach to the Job-Shop Scheduling Problem , 2008, CPAIOR.

[160]  Vittorio Maniezzo,et al.  Exact and Approximate Nondeterministic Tree-Search Procedures for the Quadratic Assignment Problem , 1999, INFORMS J. Comput..

[161]  Matthias Prandtstetter,et al.  An integer linear programming approach and a hybrid variable neighborhood search for the car sequencing problem , 2008, Eur. J. Oper. Res..