Parallel Ranking and Selection

The Winter Simulation Conference serves as the initial publication venue for many advances in ranking and selection (R&S), including the recently developed R&S procedures that exploit high-performance parallel computing. We formulate a new stylized model for representing parallel R&S procedures, and we provide an overview of existing R&S procedures under the stylized model. We also discuss why designing R&S procedures for a parallel computing platform is nontrivial and speculate on the future of parallel R&S procedures. In this chapter, “parallel computing” means multiple processors that can execute distinct simulations independently, rather than vector or array processors designed to speed up vector-matrix calculations.

[1]  Susan R. Hunter,et al.  A comparison of two parallel ranking and selection procedures , 2014, Proceedings of the Winter Simulation Conference 2014.

[2]  Philip Heidelberger,et al.  Discrete event simulations and parallel processing: statistical properties , 1988 .

[3]  Peter W. Glynn,et al.  A large deviations perspective on ordinal optimization , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[4]  Celso Leandro Palma.  MULTI-OBJECTIVE SIMULATION OPTIMIZATION ON FINITE SETS: OPTIMAL ALLOCATION VIA SCALARIZATION , 2016 .

[5]  Philip Heidelberger,et al.  Analysis of parallel replicated simulations under a completion time constraint , 1991, TOMC.

[6]  Insup Lee,et al.  Distributed Web-based simulation optimization , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[7]  Loo Hay Lee,et al.  Stochastic Simulation Optimization - An Optimal Computing Budget Allocation , 2010, System Engineering and Operations Research.

[8]  Barry L. Nelson,et al.  A fully sequential procedure for indifference-zone selection in simulation , 2001, TOMC.

[9]  Gerassimos Barlas.  Chapter 6 – GPU programming , 2015 .

[10]  Jun Luo,et al.  Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments , 2015, Oper. Res..

[11]  Barry L. Nelson,et al.  Foundations and Methods of Stochastic Simulation: A First Course , 2013 .

[12]  Enver Yücesan,et al.  Guest editors' introduction to special issue on the first INFORMS simulation society research workshop , 2010, TOMC.

[13]  Marina Schmid,et al.  Design And Analysis Of Experiments For Statistical Selection Screening And Multiple Comparisons , 2016 .

[14]  Yi Zhu,et al.  Three asymptotic regimes for ranking and selection with general sample distributions , 2016, 2016 Winter Simulation Conference (WSC).

[15]  Jürgen Branke,et al.  Selecting a Selection Procedure , 2007, Manag. Sci..

[16]  E. Jack Chen.  Using parallel and distributed computing to increase the capability of selection procedures , 2005, Proceedings of the Winter Simulation Conference, 2005..

[17]  Philip Heidelberger,et al.  Bias Properties of Budget Constrained Simulations , 1990, Oper. Res..

[18]  Joseph M. Sussman,et al.  A bayesian approach to the design of simulation experiments , 1976, WSC '76.

[19]  Dashi I. Singham,et al.  Multiple comparisons with a standard using false discovery rates , 2016, 2016 Winter Simulation Conference (WSC).

[20]  Raghu Pasupathy,et al.  Simulation Optimization: A Concise Overview and Implementation Guide , 2013 .

[21]  Jie Xu,et al.  Industrial strength COMPASS: A comprehensive algorithm and software for optimization via simulation , 2010, TOMC.

[22]  Barry L. Nelson,et al.  Indifference-Zone-Free Selection of the Best , 2016, Oper. Res..

[23]  Susan R. Hunter,et al.  Ranking and selection in a high performance computing environment , 2013, 2013 Winter Simulations Conference (WSC).

[24]  A. Tamhane.  Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons , 1995 .

[25]  Hyunbo Cho,et al.  Web Services-Based Parallel Replicated Discrete Event Simulation for Large-Scale Simulation Optimization , 2009, Simul..

[26]  Barry L. Nelson,et al.  Foundations and Methods of Stochastic Simulation , 2021, International Series in Operations Research & Management Science.

[27]  Robert D. Nowak,et al.  Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting , 2014, 2014 48th Annual Conference on Information Sciences and Systems (CISS).

[28]  S. Dalal,et al.  ALLOCATION OF OBSERVATIONS IN RANKING AND SELECTION WITH UNEQUAL VARIANCES , 1971 .

[29]  Sébastien Bubeck,et al.  Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems , 2012, Found. Trends Mach. Learn..

[30]  S. Andradóttir,et al.  Fully sequential procedures for comparing constrained systems via simulation , 2010 .

[31]  Jürgen Branke,et al.  Sequential Sampling to Myopically Maximize the Expected Value of Information , 2010, INFORMS J. Comput..

[32]  Gerassimos Barlas,et al.  Multicore and GPU Programming: An Integrated Approach , 2014 .

[33]  Susan R. Hunter,et al.  Comparing Message Passing Interface and MapReduce for large-scale parallel ranking and selection , 2015, 2015 Winter Simulation Conference (WSC).

[34]  Y. Rinott.  On two-stage selection procedures and related probability-inequalities , 1978 .

[35]  Michael C. Fu,et al.  Handbook of Simulation Optimization , 2014 .

[36]  Susan R. Hunter,et al.  Maximizing quantitative traits in the mating design problem via simulation-based Pareto estimation , 2016 .

[37]  L. Lee,et al.  Finding the non-dominated Pareto set for multi-objective simulation models , 2010 .

[38]  John S. Carson,et al.  Multiple Decision Procedures: Theory and Methodology of Selecting and Ranking Populations , 1981 .

[39]  Barry L. Nelson,et al.  Chapter 17 Selecting the Best System , 2006, Simulation.

[40]  Barry L. Nelson,et al.  Using Ranking and Selection to "Clean Up" after Simulation Optimization , 2003, Oper. Res..

[41]  Bruce W. Schmeiser,et al.  Batch Size Effects in the Analysis of Simulation Output , 1982, Oper. Res..

[42]  R. Bechhofer.  A Single-Sample Multiple Decision Procedure for Ranking Means of Normal Populations with known Variances , 1954 .

[43]  Julie L. Swann,et al.  Simple Procedures for Selecting the Best Simulated System When the Number of Alternatives is Large , 2001, Oper. Res..

[44]  V. Fabian.  Note on Anderson's Sequential Procedures with Triangular Boundary , 1974 .

[45]  Edward J. Dudewicz,et al.  Statistics in simulation: How to design for selecting the best alternative , 1976, WSC '76.

[46]  Barry L. Nelson,et al.  Discrete Optimization via Simulation Using COMPASS , 2006, Oper. Res..

[47]  Eric A. Applegate,et al.  AN INTRODUCTION TO MULTI-OBJECTIVE SIMULATION OPTIMIZATION , 2017 .

[48]  Loo Hay Lee,et al.  Stochastically Constrained Ranking and Selection via SCORE , 2014, ACM Trans. Model. Comput. Simul..

[49]  Honggang Wang,et al.  Integer-Ordered Simulation Optimization using R-SPLINE: Retrospective Search with Piecewise-Linear Interpolation and Neighborhood Enumeration , 2013, TOMC.

[50]  Chun-Hung Chen,et al.  Dynamic Sampling Allocation and Design Selection , 2016, INFORMS J. Comput..

[51]  J. Hsu.  Constrained Simultaneous Confidence Intervals for Multiple Comparisons with the Best , 1984 .

[52]  Guy Feldman,et al.  SCORE Allocations for Bi-objective Ranking and Selection , 2018, ACM Trans. Model. Comput. Simul..

[53]  E. Paulson.  A Sequential Procedure for Selecting the Population with the Largest Mean from $k$ Normal Populations , 1964 .

[54]  Susan R. Hunter,et al.  Efficient Ranking and Selection in Parallel Computing Environments , 2015, Oper. Res..

[55]  Chun-Hung Chen,et al.  Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization , 2000, Discret. Event Dyn. Syst..

[56]  David Goldsman,et al.  Ranking and selection in simulation , 1983, WSC '83.

[57]  Jun Luo,et al.  Large-scale ranking and selection using cloud computing , 2011, Proceedings of the 2011 Winter Simulation Conference (WSC).

[58]  Barry L. Nelson,et al.  Statistical screening, selection, and multiple comparison procedures in computer simulation , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[59]  Barry L. Nelson,et al.  On the Asymptotic Validity of Fully Sequential Selection Procedures for Steady-State Simulation , 2006, Oper. Res..

[60]  Ilya O. Ryzhov,et al.  On the Convergence Rates of Expected Improvement Methods , 2016, Oper. Res..

[61]  Warren B. Powell,et al.  A Knowledge-Gradient Policy for Sequential Information Collection , 2008, SIAM J. Control. Optim..

[62]  Warren B. Powell,et al.  The Knowledge-Gradient Algorithm for Sequencing Experiments in Drug Discovery , 2011, INFORMS J. Comput..