Fully sequential procedures for comparing constrained systems via simulation

We consider the problem of finding the system with the best primary performance measure among a finite number of simulated systems in the presence of a stochastic constraint on a single real‐valued secondary performance measure. Solving this problem requires the identification and removal from consideration of infeasible systems (Phase I) and of systems whose primary performance measure is dominated by that of other feasible systems (Phase II). We use indifference zones in both phases and consider two approaches, namely, carrying out Phases I and II sequentially and carrying out Phases I and II simultaneously, and we provide specific example procedures of each type. We present theoretical results guaranteeing that our approaches (general and specific, sequential and simultaneous) yield the best system with at least a prespecified probability, and we provide a portion of an extensive numerical study aimed at evaluating and comparing the performance of our approaches. The experimental results show that both new procedures are useful for constrained ranking and selection, with neither procedure showing uniform superiority over the other.© 2010 Wiley Periodicals, Inc. Naval Research Logistics, 2010

[1]  R. Bechhofer.  A Single-Sample Multiple Decision Procedure for Ranking Means of Normal Populations with known Variances , 1954 .

[2]  E. Paulson.  A Sequential Procedure for Selecting the Population with the Largest Mean from $k$ Normal Populations , 1964 .

[3]  V. Fabian.  Note on Anderson's Sequential Procedures with Triangular Boundary , 1974 .

[4]  Y. Rinott.  On two-stage selection procedures and related probability-inequalities , 1978 .

[5]  I. Johnstone,et al.  ASYMPTOTICALLY OPTIMAL PROCEDURES FOR SEQUENTIAL ADAPTIVE SELECTION OF THE BEST OF SEVERAL NORMAL MEANS , 1982 .

[6]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[7]  T. Santer,et al.  Designing Experiments for Selecting a Normal Population with a Large Mean and a Small Variance , 1983 .

[8]  M. Hartmann.  An improvement on paulson s sequential ranking procedure , 1988 .

[9]  M. Hartmann.  An improvement on paulson's procedure for selecting the poprlation with the largest mean from k normal populations with a common unknown variance , 1991 .

[10]  Chun-Hung Chen.  A lower bound for the correct subset-selection probability and its application to discrete-event system simulations , 1996, IEEE Trans. Autom. Control..

[11]  Stephen E. Chick.  Selecting the best system: a decision-theoretic approach , 1997, WSC '97.

[12]  Chun-Hung Chen,et al.  Computing efforts allocation for ordinal optimization and discrete event simulation , 2000, IEEE Trans. Autom. Control..

[13]  Barry L. Nelson,et al.  A fully sequential procedure for indifference-zone selection in simulation , 2001, TOMC.

[14]  Douglas J. Morrice,et al.  A Multiple Attribute Utility Theory Approach to Ranking and Selection , 2001, Manag. Sci..

[15]  Stephen E. Chick,et al.  New Procedures to Select the Best Simulated System Using Common Random Numbers , 2001, Manag. Sci..

[16]  Julie L. Swann,et al.  Simple Procedures for Selecting the Best Simulated System When the Number of Alternatives is Large , 2001, Oper. Res..

[17]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[18]  Barry L. Nelson,et al.  Ranking and Selection for Steady-State Simulation: Procedures and Perspectives , 2002, INFORMS J. Comput..

[19]  Barry L. Nelson,et al.  Using Ranking and Selection to "Clean Up" after Simulation Optimization , 2003, Oper. Res..

[20]  Seong-Hee Kim,et al.  Performance of variance updating ranking and selection procedures , 2005, Proceedings of the Winter Simulation Conference, 2005..

[21]  Sigrún Andradóttir,et al.  Finding the best in the presence of a stochastic constraint , 2005, Proceedings of the Winter Simulation Conference, 2005..

[22]  Lih-Yuan Deng,et al.  Efficient and portable multiple recursive generators of large order , 2005, TOMC.

[23]  Barry L. Nelson,et al.  The tradeoff between sampling and switching: New sequential procedures for indifference-zone selection , 2005 .

[24]  David Goldsman,et al.  PERFORMANCE OF VARIANCE UPDATING PROCEDURES ON VARIOUS DATA , 2005 .

[25]  Seong-Hee Kim,et al.  Comparison with a standard via fully sequential procedures , 2005, TOMC.

[26]  Barry L. Nelson,et al.  On the Asymptotic Validity of Fully Sequential Selection Procedures for Steady-State Simulation , 2006, Oper. Res..

[27]  Barry L. Nelson,et al.  A sequential procedure for neighborhood selection-of-the-best in optimization via simulation , 2006, Eur. J. Oper. Res..

[28]  Demet Batur,et al.  Fully sequential selection procedures with a parabolic boundary , 2006 .