Prediction of Parallel Speed-Ups for Las Vegas Algorithms

We propose a probabilistic model for the parallel execution of Las Vegas algorithms, i.e. randomized algorithms whose runtime might vary from one execution to another, even with the same input. This model aims at predicting the parallel performances (i.e. speedups) by analyzing the runtime distribution of the sequential runs of the algorithm. Then, we study in practice the case of a particular Las Vegas algorithm for combinatorial optimization on three classical problems, and compare the model with an actual parallel implementation up to 256 cores. We show that the prediction can be accurate, matching the actual speedups very well up to 100 parallel cores and then with a deviation of about 20% up to 256 cores.

[1]  J.P. Costas,et al.  A study of a class of detection waveforms having nearly ideal range—Doppler ambiguity properties , 1983, Proceedings of the IEEE.

[2]  Francesco Iorio,et al.  The enumeration of Costas arrays of order 28 , 2010, ITW.

[3]  P. Codognet,et al.  Musical constraint satisfaction problems solved with adaptive search , 2004, Soft Comput..

[4]  El-Ghazali Talbi,et al.  Local Search Algorithms on Graphics Processing Units. A Case Study: The Permutation Perceptron Problem , 2010, EvoCOP.

[5]  Pierre Hansen,et al.  Cooperative Parallel Variable Neighborhood Search for the p-Median , 2004, J. Heuristics.

[6]  David,et al.  [Wiley Series in Probability and Statistics] Order Statistics (David/Order Statistics) || Basic Distribution Theory , 2003 .

[8]  Philippe Codognet,et al.  Yet Another Local Search Method for Constraint Solving , 2001, SAGA.

[9]  Toshihide Ibaraki,et al.  Metaheuristics : progress as real problem solvers , 2005 .

[10]  Francesco Iorio,et al.  Results of the Enumeration of Costas Arrays of Order $27$ , 2008, IEEE Transactions on Information Theory.

[11]  Teodor Gabriel Crainic,et al.  Introduction to the Special Issue on Parallel Meta-Heuristics , 2002, J. Heuristics.

[12]  Shen Lin.  Computer solutions of the traveling salesman problem , 1965 .

[13]  K. Drakakis,et al.  A review of Costas arrays , 2006 .

[14]  Pascal Van Hentenryck.  Constraint satisfaction in logic programming , 1989, Logic programming.

[15]  Bart Selman,et al.  Algorithm portfolios , 2001, Artif. Intell..

[16]  J.K. Beard,et al.  Costas array generation and search methodology , 2007, IEEE Transactions on Aerospace and Electronic Systems.

[17]  Philippe Codognet,et al.  Performance analysis of parallel constraint-based local search , 2012, PPoPP '12.

[18]  Saralees Nadarajah,et al.  Explicit expressions for moments of order statistics , 2008 .

[19]  S. Golomb,et al.  Constructions and properties of Costas arrays , 1984, Proceedings of the IEEE.

[20]  Jakub Marecek,et al.  Handbook of Approximation Algorithms and Metaheuristics , 2010, Comput. J..

[21]  Toby Walsh,et al.  CSPLIB: A Benchmark Library for Constraints , 1999, CP.

[22]  L. Babai.  Monte-Carlo algorithms in graph isomorphism testing , 2006 .

[23]  P. Rousseeuw,et al.  Wiley Series in Probability and Mathematical Statistics , 2005 .

[24]  Celso C. Ribeiro,et al.  Probability Distribution of Solution Time in GRASP: An Experimental Investigation , 2002, J. Heuristics.

[25]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[26]  C.J.H. Mann,et al.  Handbook of Approximation: Algorithms and Metaheuristics , 2008 .

[27]  Jin-Kao Hao,et al.  A General Approach for Constraint Solving by Local Search , 2004, J. Math. Model. Algorithms.

[28]  Franck Cappello,et al.  Grid'5000: A Large Scale And Highly Reconfigurable Experimental Grid Testbed , 2006, Int. J. High Perform. Comput. Appl..

[29]  Panos M. Pardalos,et al.  Parallel Search for Combinatorial Optimization: Genetic Algorithms, Simulated Annealing, Tabu Search and GRASP , 1995, IRREGULAR.

[30]  Stephen Wolfram,et al.  The Mathematica book, 5th Edition , 2003 .

[31]  Celso C. Ribeiro,et al.  TTT plots: a perl program to create time-to-target plots , 2007, Optim. Lett..

[32]  Christian Bessiere,et al.  Constraint Propagation , 2006, Handbook of Constraint Programming.

[33]  James K. Beard,et al.  Costas array search technique that maximizes backtrack and symmetry exploitation , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[34]  F. James.  Statistical Methods in Experimental Physics , 1973 .

[35]  Philippe Codognet,et al.  An Efficient Library for Solving CSP with Local Search , 2003 .

[36]  Philippe Codognet,et al.  Parallel Local Search for the Costas Array Problem , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum.

[37]  Solomon W. Golomb,et al.  Algebraic Constructions for Costas Arrays , 1984, J. Comb. Theory, Ser. A.

[38]  Herbert A. David,et al.  Order Statistics, Third Edition , 2003, Wiley Series in Probability and Statistics.

[39]  Yves Crama,et al.  Local Search in Combinatorial Optimization , 2018, Artificial Neural Networks.

[40]  Steven Minton,et al.  Minimizing Conflicts: A Heuristic Repair Method for Constraint Satisfaction and Scheduling Problems , 1992, Artif. Intell..

[41]  Emile H. L. Aarts,et al.  Parallel local search , 1995, J. Heuristics.