Learning and Intelligent Optimization

In view of the increasing importance of hardware parallelism, a natural extension of per-instance algorithm selection is to select a set of algorithms to be run in parallel on a given problem instance, based on features of that instance. Here, we explore how existing algorithm selection techniques can be effectively parallelized. To this end, we leverage the machine learning models used by existing sequential algorithm selectors, such as 3S , ISAC , SATzilla and ME-ASP, and modify their selection procedures to produce a ranking of the given candidate algorithms; we then select the top n algorithms under this ranking to be run in parallel on n processing units. Furthermore, we adapt the pre-solving schedules obtained by aspeed to be effective in a parallel setting with different time budgets for each processing unit. Our empirical results demonstrate that, using 4 processing units, the best of our methods achieves a 12-fold average speedup over the best single solver on a broad set of challenging scenarios from the algorithm selection library.

[1]  Sébastien Vérel,et al.  Local Optima Networks of the Permutation Flow-Shop Problem , 2013, Artificial Evolution.

[2]  Colin R. Reeves,et al.  A genetic algorithm for flowshop sequencing , 1995, Comput. Oper. Res..

[3]  Sanja Petrovic,et al.  HyFlex: A Benchmark Framework for Cross-Domain Heuristic Search , 2011, EvoCOP.

[4]  E. B. Dynkin,et al.  Sätze Und Aufgaben Über Markoffsche Prozesse , 1969 .

[5]  Stephen S. Yau,et al.  Toward Development of Adaptive Service-Based Software Systems , 2009, IEEE Transactions on Services Computing.

[6]  Jan Karel Lenstra,et al.  Complexity of machine scheduling problems , 1975 .

[7]  Thomas Stützle,et al.  Local search algorithms for combinatorial problems - analysis, improvements, and new applications , 1999, DISKI.

[8]  Thomas Stützle,et al.  F-Race and Iterated F-Race: An Overview , 2010, Experimental Methods for the Analysis of Optimization Algorithms.

[9]  P. Stadler Landscapes and their correlation functions , 1996 .

[10]  Thomas Stützle,et al.  A review of metrics on permutations for search landscape analysis , 2007, Comput. Oper. Res..

[11]  Aldy Gunawan,et al.  A Hybrid Algorithm for the Quadratic Assignment Problem , 2008, CSC.

[12]  Laetitia Vermeulen-Jourdan,et al.  Fitness Landscape Analysis and Metaheuristics Efficiency , 2013, J. Math. Model. Algorithms.

[13]  David Lindley,et al.  Dynamic Programming and Decision Theory , 1961 .

[14]  Zhi Yuan,et al.  Automated Parameter Tuning Framework for Heterogeneous and Large Instances: Case Study in Quadratic Assignment Problem , 2013, LION.

[15]  Marek Petrik,et al.  Learning parallel portfolios of algorithms , 2006, Annals of Mathematics and Artificial Intelligence.

[16]  F Thomas Bruss A Unified Approach to a Class of Best Choice Problems with an Unknown Number of Options , 1984 .

[17]  Éric D. Taillard,et al.  Benchmarks for basic scheduling problems , 1993 .

[18]  Lars Kotthoff,et al.  LLAMA: Leveraging Learning to Automatically Manage Algorithms , 2013, ArXiv.

[19]  Sébastien Vérel,et al.  Scuba search: when selection meets innovation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[20]  Jatinder N. D. Gupta,et al.  Scheduling identical parallel machines to minimize total tardiness , 2008 .

[21]  Kang Zhang,et al.  A hybrid discrete differential evolution algorithm to minimise total tardiness on identical parallel machines , 2013, Int. J. Comput. Integr. Manuf..

[22]  Kevin Leyton-Brown,et al.  Hydra: Automatically Configuring Algorithms for Portfolio-Based Selection , 2010, AAAI.

[23]  Pierre Hansen,et al.  Variable Neighbourhood Search , 2003 .

[24]  Oliver Skroch Multi-criteria Service Selection with Optimal Stopping in Dynamic Service-Oriented Systems , 2010, ICDCIT.

[25]  E. Weinberger,et al.  Correlated and uncorrelated fitness landscapes and how to tell the difference , 1990, Biological Cybernetics.

[26]  Tad Hogg,et al.  An Economics Approach to Hard Computational Problems , 1997, Science.

[27]  Lindawati,et al.  Fine-Tuning Algorithm Parameters Using the Design of Experiments Approach , 2011, LION.

[28]  Sébastien Vérel,et al.  NILS: A Neutrality-Based Iterated Local Search and Its Application to Flowshop Scheduling , 2011, EvoCOP.

[29]  Isaac M. Sonin,et al.  The Best Choice Problem for a Random Number of Objects , 1973 .

[30]  Bart Selman,et al.  Algorithm portfolios , 2001, Artif. Intell..

[31]  Ling Wang,et al.  A hybrid particle swarm optimization for parallel machine total tardiness scheduling , 2010 .

[32]  Thomas Stützle,et al.  Automatic Algorithm Configuration Based on Local Search , 2007, AAAI.

[33]  Gerhard Reinelt,et al.  TSPLIB - A Traveling Salesman Problem Library , 1991, INFORMS J. Comput..

[34]  John R. Rice,et al.  The Algorithm Selection Problem , 1976, Adv. Comput..

[35]  Yuri Malitsky,et al.  ISAC - Instance-Specific Algorithm Configuration , 2010, ECAI.

[36]  Rubén Ruiz,et al.  A comprehensive review and evaluation of permutation flowshop heuristics to minimize flowtime , 2013, Comput. Oper. Res..