Generating Applicable Synthetic Instances for Branch Problems

Generating valid synthetic instances for branch problems—those that contain a core problem like knapsack or graph coloring, but add several complications—is hard. It is even harder to generate instances that are applicable to the specific goals of an experiment and help to support the claims made. This paper presents a methodology for tuning instance generators of branch problems so that synthetic instances are similar to real ones and are capable of eliciting different behaviors from solvers. A statistic is proposed to summarize the applicability of instances for drawing a valid conclusion. The methodology is demonstrated on the Udine timetabling problem. Examples and the necessary cyberinfrastructure are available as a project from Computational Infrastructure for Operations Research (COIN-OR).

[1]  Richard J. Anderson,et al.  The role of experiment in the theory of algorithms , 1999, Data Structures, Near Neighbor Searches, and Methodology.

[2]  R. Lougee-Heimer,et al.  The Common Optimization INterface for Operations Research: Promoting open-source software in the operations research community , 2003 .

[3]  Marc E. Posner,et al.  Generating Experimental Data for Computational Testing with Machine Scheduling Applications , 2001, Oper. Res..

[4]  Edmund K. Burke,et al.  Decomposition, reformulation, and diving in university course timetabling , 2009, Comput. Oper. Res..

[5]  Gary J. Koehler,et al.  Conditions that Obviate the No-Free-Lunch Theorems for Optimization , 2007, INFORMS J. Comput..

[6]  Daniel Brélaz,et al.  New methods to color the vertices of a graph , 1979, CACM.

[7]  Marc E. Posner,et al.  Performance Prediction and Preselection for Optimization and Heuristic Solution Procedures , 2007, Oper. Res..

[8]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[9]  Rémi Monasson,et al.  Determining computational complexity from characteristic ‘phase transitions’ , 1999, Nature.

[10]  Thomas Bartz-Beielstein,et al.  Experimental Methods for the Analysis of Optimization Algorithms , 2010 .

[11]  Yoav Shoham,et al.  Empirical hardness models: Methodology and a case study on combinatorial auctions , 2009, JACM.

[12]  Assaf Naor,et al.  Rigorous location of phase transitions in hard optimization problems , 2005, Nature.

[13]  John E. Beasley,et al.  OR-Library: Distributing Test Problems by Electronic Mail , 1990 .

[14]  Robert Schaefer.  Parallel Problem Solving from Nature - PPSN XI, 11th International Conference, Kraków, Poland, September 11-15, 2010. Proceedings, Part II , 2010, PPSN.

[15]  É. Taillard.  COMPARISON OF ITERATIVE SEARCHES FOR THE QUADRATIC ASSIGNMENT PROBLEM. , 1995 .

[16]  Charles H. Reilly,et al.  Synthetic Optimization Problem Generation: Show Us the Correlations! , 2009, INFORMS J. Comput..

[17]  Nello Cristianini,et al.  Support vector machines , 2009 .

[18]  Thomas Bartz-Beielstein,et al.  Design and Analysis of Optimization Algorithms Using Computational Statistics , 2004 .

[19]  John N. Hooker,et al.  Testing heuristics: We have it all wrong , 1995, J. Heuristics.

[20]  A. Rényii,et al.  ON A PROBLEM OF GRAPH THEORY , 1966 .

[21]  Marc E. Posner,et al.  The Generation of Experimental Data for Computational Testing in Optimization , 2010, Experimental Methods for the Analysis of Optimization Algorithms.

[22]  A. L. Tuson,et al.  Problem difficulty of real instances of convoy planning , 2005, J. Oper. Res. Soc..

[23]  David S. Johnson,et al.  A theoretician's guide to the experimental analysis of algorithms , 1999, Data Structures, Near Neighbor Searches, and Methodology.

[24]  Tomás Müller,et al.  ITC2007 solver description: a hybrid approach , 2009, Ann. Oper. Res..

[25]  Bernard M. E. Moret,et al.  DIMACS Series in Discrete Mathematics and Theoretical Computer Science Towards a Discipline of Experimental Algorithmics , 2022 .

[26]  Jano I. van Hemert,et al.  Discovering the suitability of optimisation algorithms by learning from evolved instances , 2011, Annals of Mathematics and Artificial Intelligence.

[27]  David W. Corne,et al.  Optimisation and Generalisation: Footprints in Instance Space , 2010, PPSN.

[28]  Krzysztof Socha,et al.  Hardness Prediction for the University Course Timetabling Problem , 2004, EvoCOP.

[29]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[30]  Catherine C. McGeoch.  A bibliography of algorithm experimentation , 1999, Data Structures, Near Neighbor Searches, and Methodology.

[31]  Nong Ye,et al.  The Handbook of Data Mining , 2003 .

[32]  Edmund K. Burke,et al.  Enrollment generators, clustering and chromatic numbers , 2008 .

[33]  Edmund K. Burke,et al.  A supernodal formulation of vertex colouring with applications in course timetabling , 2007, Ann. Oper. Res..

[34]  David R. Wood,et al.  An algorithm for finding a maximum clique in a graph , 1997, Oper. Res. Lett..

[35]  Toshihide Ibaraki,et al.  An Improved Tabu Search Method For The Weighted Constraint Satisfaction Problem , 2001 .

[36]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[37]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[38]  Douglas Thain,et al.  Distributed computing in practice: the Condor experience , 2005, Concurr. Pract. Exp..

[39]  Kate Smith-Miles,et al.  Generalising Algorithm Performance in Instance Space: A Timetabling Case Study , 2011, LION.

[40]  Charles H. Reilly,et al.  The Effects of Coefficient Correlation Structure in Two-Dimensional Knapsack Problems on Solution Procedure Performance , 2000 .

[41]  Kate Smith-Miles,et al.  Measuring algorithm footprints in instance space , 2012, 2012 IEEE Congress on Evolutionary Computation.

[42]  Kate Smith-Miles,et al.  Measuring instance difficulty for combinatorial optimization problems , 2012, Comput. Oper. Res..

[43]  K. A. Ariyawansa,et al.  On a New Collection of Stochastic Linear Programming Test Problems , 2004, INFORMS J. Comput..