Computational Comparison of Metaheuristics

Metaheuristics are truly diverse in nature—under the overarching theme of performing operations to escape local optima, algorithms as different as ant colony optimization, tabu search, harmony search, and genetic algorithms have emerged. Due to the unique functionality of each type of metaheuristic, the computational comparison of metaheuristics is in many ways more difficult than other algorithmic comparisons. In this chapter, we discuss techniques for the meaningful computational comparison of metaheuristics. We discuss how to create and classify instances in a new testbed and how to make sure other researchers have access to these test instances for future metaheuristic comparisons. In addition, we discuss the disadvantages of large parameter sets and how to measure complicated parameter interactions in a metaheuristic’s parameter space. Finally, we explain how to compare metaheuristics in terms of both solution quality and runtime and how to compare parallel metaheuristics.

[1]  Alice Paul,et al.  Prize-Collecting TSP with a Budget Constraint , 2017, ESA.

[2]  Yu Xue,et al.  Combining Wang-Landau sampling algorithm and heuristics for solving the unequal-area dynamic facility layout problem , 2017, Eur. J. Oper. Res..

[3]  Bruce L. Golden,et al.  A Parallel Algorithm for the Vehicle Routing Problem , 2011, INFORMS J. Comput..

[4]  R. Plackett,et al.  THE DESIGN OF OPTIMUM MULTIFACTORIAL EXPERIMENTS , 1946 .

[5]  Jack J. Dongarra,et al.  Performance of various computers using standard linear equations software in a FORTRAN environment , 1988, CARN.

[6]  Kalyanmoy Deb,et al.  Understanding Interactions among Genetic Algorithm Parameters , 1998, FOGA.

[7]  Madalina M. Drugan.  Generating QAP instances with known optimum solution and additively decomposable cost function , 2015, J. Comb. Optim..

[8]  Michel Gendreau,et al.  A tabu search heuristic for the undirected selective travelling salesman problem , 1998, Eur. J. Oper. Res..

[9]  Bruce L. Golden,et al.  The effective application of a new approach to the generalized orienteering problem , 2010, J. Heuristics.

[10]  Michel Gendreau,et al.  Heuristics for multi-attribute vehicle routing problems: A survey and synthesis , 2013, Eur. J. Oper. Res..

[11]  Nantiwat Pholdee,et al.  Comparative performance of meta-heuristic algorithms for mass minimisation of trusses with dynamic constraints , 2014, Adv. Eng. Softw..

[12]  Gerhard Reinelt,et al.  TSPLIB - A Traveling Salesman Problem Library , 1991, INFORMS J. Comput..

[13]  Taimoor Akhtar,et al.  Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection , 2016, J. Glob. Optim..

[14]  Cecilia R. Aragon,et al.  Optimization by Simulated Annealing: An Experimental Evaluation; Part II, Graph Coloring and Number Partitioning , 1991, Oper. Res..

[15]  Bruce L. Golden,et al.  The open vehicle routing problem: Algorithms, large-scale test problems, and computational results , 2007, Comput. Oper. Res..

[16]  Ravindra K. Ahuja,et al.  Use of Representative Operation Counts in Computational Testing of Algorithms , 1996, INFORMS J. Comput..

[17]  George C. Runger,et al.  Using Experimental Design to Find Effective Parameter Settings for Heuristics , 2001, J. Heuristics.

[18]  Kate Smith-Miles,et al.  Measuring instance difficulty for combinatorial optimization problems , 2012, Comput. Oper. Res..

[19]  Zeger Degraeve,et al.  Meta-heuristics for dynamic lot sizing: A review and comparison of solution approaches , 2004, Eur. J. Oper. Res..

[20]  Bryant A. Julstrom,et al.  An effective genetic algorithm for the minimum-label spanning tree problem , 2006, GECCO '06.

[21]  Richard M. Karp,et al.  The Traveling-Salesman Problem and Minimum Spanning Trees , 1970, Oper. Res..

[22]  Cecilia R. Aragon,et al.  Optimization by Simulated Annealing: An Experimental Evaluation; Part I, Graph Partitioning , 1989, Oper. Res..

[23]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[24]  Ravindra K. Ahuja,et al.  Computational investigations of maximum flow algorithms , 1997 .

[25]  Dirk Thierens,et al.  GAMBIT: A Parameterless Model-Based Evolutionary Algorithm for Mixed-Integer Problems , 2018, Evolutionary Computation.

[26]  Enrique Alba,et al.  Multi-objective optimization using metaheuristics: non-standard algorithms , 2012, Int. Trans. Oper. Res..

[27]  Richard M. Karp,et al.  The traveling-salesman problem and minimum spanning trees: Part II , 1971, Math. Program..

[28]  Ping Chen,et al.  A novel approach to solve the split delivery vehicle routing problem , 2017, Int. Trans. Oper. Res..

[29]  Michael H. Goldwasser,et al.  Data Structures, Near Neighbor Searches, and Methodology: Fifth and Sixth DIMACS Implementation Challenges, Proceedings of a DIMACS Workshop, USA, 1999 , 2002, Data Structures, Near Neighbor Searches, and Methodology.

[30]  Andrew Davison,et al.  Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers , 1995 .

[31]  Iain Dunning,et al.  What Works Best When? A Systematic Evaluation of Heuristics for Max-Cut and QUBO , 2018, INFORMS J. Comput..

[32]  Dirk Sudholt,et al.  Parallel Evolutionary Algorithms , 2015, Handbook of Computational Intelligence.

[33]  Reha Uzsoy,et al.  Experimental Evaluation of Heuristic Optimization Algorithms: A Tutorial , 2001, J. Heuristics.

[34]  Juan José Miranda Bront,et al.  A meta-heuristic based goal-selection strategy for mobile robot search in an unknown environment , 2017, Comput. Oper. Res..

[35]  Matthew J. Saltzman,et al.  Statistical Analysis of Computational Tests of Algorithms and Heuristics , 2000, INFORMS J. Comput..

[36]  Daniel Sánchez,et al.  Fractal: An execution model for fine-grain nested speculative parallelism , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[37]  Andrzej Jaszkiewicz,et al.  Do multiple-objective metaheuristics deliver on their promises? A computational experiment on the set-covering problem , 2003, IEEE Trans. Evol. Comput..

[38]  Enrique Alba,et al.  Parallel metaheuristics: recent advances and new trends , 2012, Int. Trans. Oper. Res..

[39]  Diego Klabjan,et al.  Algorithms for Generalized Clusterwise Linear Regression , 2017, INFORMS J. Comput..

[40]  Bruce L. Golden,et al.  Very large-scale vehicle routing: new test problems, algorithms, and results , 2005, Comput. Oper. Res..

[41]  Kate Smith-Miles,et al.  Towards objective measures of algorithm performance across instance space , 2014, Comput. Oper. Res..

[42]  Leslie Pérez Cáceres,et al.  The irace package: Iterated racing for automatic algorithm configuration , 2016 .

[43]  Bruce L. Golden,et al.  A record-to-record travel algorithm for solving the heterogeneous fleet vehicle routing problem , 2007, Comput. Oper. Res..

[44]  B. Golden,et al.  Models and branch-and-cut algorithms for pickup and delivery problems with time windows , 2007 .

[45]  Bruce L. Golden,et al.  A one-parameter genetic algorithm for the minimum labeling spanning tree problem , 2005, IEEE Transactions on Evolutionary Computation.

[46]  Bruce L. Golden,et al.  The min-max multi-depot vehicle routing problem: heuristics and computational results , 2015, J. Oper. Res. Soc..

[47]  Jano I. van Hemert,et al.  Discovering the suitability of optimisation algorithms by learning from evolved instances , 2011, Annals of Mathematics and Artificial Intelligence.

[48]  David W. Corne,et al.  Optimisation and Generalisation: Footprints in Instance Space , 2010, PPSN.

[49]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[50]  Matteo Fischetti,et al.  A Branch-and-Cut Algorithm for the Symmetric Generalized Traveling Salesman Problem , 1997, Oper. Res..

[51]  Mehrdad Tamiz,et al.  Multi-objective meta-heuristics: An overview of the current state-of-the-art , 2002, Eur. J. Oper. Res..

[52]  Bruce L. Golden,et al.  Algorithms and solutions to multi-level vehicle routing problems , 1993 .

[53]  James P. Kelly,et al.  A Network Flow-Based Tabu Search Heuristic for the Vehicle Routing Problem , 1996, Transp. Sci..

[54]  Carlos A. Coello Coello,et al.  Evolutionary multi-objective optimization: a historical view of the field , 2006, IEEE Comput. Intell. Mag..

[55]  Kate Smith-Miles,et al.  Generating new test instances by evolving in instance space , 2015, Comput. Oper. Res..

[56]  Enrique Alba,et al.  Parallel Genetic Algorithms , 2020, Studies in Computational Intelligence.

[57]  Florent Krzakala,et al.  Threshold values, stability analysis and high-q asymptotics for the coloring problem on random graphs , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[58]  Aric Hagberg,et al.  Exploring Network Structure, Dynamics, and Function using NetworkX , 2008, Proceedings of the Python in Science Conference.

[59]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[60]  S. Raghavan,et al.  The Multilevel Capacitated Minimum Spanning Tree Problem , 2006, INFORMS J. Comput..

[61]  Gilbert Laporte,et al.  Exact and heuristic algorithms for the Hamiltonian p-median problem , 2016, Eur. J. Oper. Res..

[62]  Michel Gendreau,et al.  The traveling salesman problem with time-dependent service times , 2016, Eur. J. Oper. Res..

[63]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[64]  J. Mark Bull,et al.  Benchmarking Java against C and Fortran for scientific applications , 2001, JGI '01.