Ranking and Selection for Steady-State Simulation: Procedures and Perspectives

We present and evaluate three ranking-and-selection procedures for use in steady-state simulation experiments when the goal is to find which among a finite number of alternative systems has the largest or smallest long-run average performance. All three procedures extend existing methods for independent and identically normally distributed observations to general stationary output processes, and all procedures are sequential. We also provide our thoughts about the evaluation of simulation design and analysis procedures, and illustrate these concepts in our evaluation of the new procedures.

[1]  Barry L. Nelson,et al.  Ranking and selection for steady-state simulation , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[2]  David Goldsman,et al.  Selection procedures with standardized time series variance estimators , 1999, WSC '99.

[3]  Natalie M. Steiger,et al.  Output analysis: ASAP2: an improved batch means procedure for simulation output analysis , 2002, WSC '02.

[4]  David Goldsman,et al.  Ranking and selection in simulation , 1983, WSC '83.

[5]  Halim Damerdji,et al.  Mean-Square Consistency of the Variance Estimator in Steady-State Simulation Output Analysis , 1995, Oper. Res..

[6]  L. Schruben,et al.  Properties of standardized time series weighted area variance estimators , 1990 .

[7]  Bruce W. Schmeiser,et al.  Inverse-Transformation Algorithms For Some Common Stochastic Processes , 1989 .

[8]  Lee W. Schruben,et al.  Confidence Interval Estimation Using Standardized Time Series , 1983, Oper. Res..

[9]  Y. Rinott.  On two-stage selection procedures and related probability-inequalities , 1978 .

[10]  Ward Whitt,et al.  Estimating the asymptotic variance with batch means , 1991, Oper. Res. Lett..

[11]  David Goldsman.  Ranking and selection procedures using standardized time series , 1985, WSC '85.

[12]  Marvin K. Nakayama,et al.  Two-stage procedures for multiple comparisons with a control in steady-state simulations , 1996, Winter Simulation Conference.

[13]  Barry L. Nelson,et al.  A fully sequential procedure for indifference-zone selection in simulation , 2001, TOMC.

[14]  James R. Wilson,et al.  Restricted Subset Selection Procedures for Simulation , 1989, Oper. Res..

[15]  A. Tamhane.  Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons , 1995 .

[16]  B. Schmeiser,et al.  Optimal mean-squared-error batch sizes , 1995 .

[17]  Marvin K. Nakayama,et al.  Multiple-comparison procedures for steady-state simulations , 1997 .

[18]  Marvin K. Nakayama.  Selecting the best system in steady-state simulations using batch means , 1995, WSC '95.

[19]  Halim Damerdji,et al.  Strong Consistency of the Variance Estimator in Steady-State Simulation Output Analysis , 1994, Math. Oper. Res..

[20]  David Goldsman,et al.  Consistency of several variants of the standardized time series area variance estimator , 1995 .

[21]  George S. Fishman,et al.  An Implementation of the Batch Means Method , 1997, INFORMS J. Comput..

[22]  David Goldsman,et al.  Large-Sample Results for Batch Means , 1997 .

[23]  Marvin K. Nakayama,et al.  Two-stage multiple-comparison procedures for steady-state simulations , 1999, TOMC.

[24]  Bruce W. Schmeiser,et al.  Overlapping batch means: something for nothing? , 1984, WSC '84.