Determining Whether a Problem Characteristic Affects Heuristic Performance

This chapter presents a rigorous Design of Experiments (DOE) approach for determining whether a problem characteristic affects the performance of a heuristic. Specifically, it reports a study on the effect of the cost matrix standard deviation of symmetric Travelling Salesman Problem (TSP) instances on the performance of Ant Colony Optimisation (ACO) heuristics. Results demonstrate that for a given instance size, an increase in the standard deviation of the cost matrix of instances results in an increase in the difficulty of the instances. This implies that for ACO, it is insufficient to report results on problems classified only by problem size, as has been commonly done in most ACO research to date. Some description of the cost matrix distribution is also required when attempting to explain and predict the performance of these heuristics on the TSP. The study should serve as a template for similar investigations with other problems and other heuristics.

[1]  Peter C. Cheeseman,et al.  Where the Really Hard Problems Are , 1991, IJCAI.

[2]  Keld Helsgaun,et al.  An effective implementation of the Lin-Kernighan traveling salesman heuristic , 2000, Eur. J. Oper. Res..

[3]  Gerhard Reinelt,et al.  TSPLIB - A Traveling Salesman Problem Library , 1991, INFORMS J. Comput..

[4]  Mauro Birattari.  The Problem of Tuning Metaheuristics , 2005 .

[5]  Daniel Kudenko,et al.  Analyzing heuristic performance with response surface models: prediction, optimization and robustness , 2007, GECCO '07.

[6]  Enda Ridge,et al.  Design of Experiments for the Tuning of Optimisation Algorithms , 2007 .

[7]  A. E. Eiben,et al.  A critical note on experimental research methodology in EC , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[8]  Daniel Kudenko,et al.  Tuning the Performance of the MMAS Heuristic , 2007, SLS.

[9]  Jørgen Bang-Jensen,et al.  Mixed Models for the Analysis of Local Search Components , 2007, SLS.

[10]  Marco Dorigo,et al.  Model-Based Search for Combinatorial Optimization: A Comparative Study , 2002, PPSN.

[11]  Daniel Kudenko,et al.  Screening the parameters affecting heuristic performance , 2007, GECCO '07.

[12]  Thomas Stützle,et al.  MAX-MIN Ant System , 2000, Future Gener. Comput. Syst..

[13]  John N. Hooker,et al.  Testing heuristics: We have it all wrong , 1995, J. Heuristics.

[14]  William J. Cook,et al.  Implementing the Dantzig-Fulkerson-Johnson algorithm for large traveling salesman problems , 2003, Math. Program..

[15]  N. Biggs.  THE TRAVELING SALESMAN PROBLEM A Guided Tour of Combinatorial Optimization , 1986 .

[16]  Ritu Gupta,et al.  Statistical exploratory analysis of genetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[17]  David S. Johnson,et al.  A theoretician's guide to the experimental analysis of algorithms , 1999, Data Structures, Near Neighbor Searches, and Methodology.

[18]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[20]  S.J.J. Smith,et al.  Empirical Methods for Artificial Intelligence , 1995 .

[21]  Thomas Stützle,et al.  An Analysis of the Hardness of TSP Instances for Two High-performance Algorithms , 2005 .

[22]  Jano I. van Hemert,et al.  Property Analysis of Symmetric Travelling Salesman Problem Instances Acquired Through Evolution , 2005, EvoCOP.

[23]  Edward Curry,et al.  A roadmap of nature-inspired systems research and development , 2007, Multiagent Grid Syst..

[24]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[25]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[26]  Eitan Zemel.  Measuring the Quality of Approximate Solutions to Zero-One Programming Problems , 1981, Math. Oper. Res..