A tournament framework for the ranking and selection problem

A tournament can be broadly defined as a procedure that ranks agents, where they exhibit their performance in a noisy environment. By observing the agents' performance, the organizer computes their ranking and rewards them according to the revealed ranking. The organizer's challenge is therefore to determine the optimal tournament format that identifies the best agent in the most effective fashion. Tournaments thus provide a natural framework for ranking and selection (R&S) via simulation, which represents a set of approaches developed to complement the modeling flexibility of simulation with the efficiency of statistical techniques for effective decision making. In this paper, following the introduction of a general framework to represent various tournament formats and to assess their predictive power, we will report preliminary experimental results on the effectiveness of tournaments in identifying the best simulated system with the desired probability of correct selection in the presence of costs.

[1]  Chun-Hung Chen,et al.  New development of optimal computing budget allocation for discrete event simulation , 1997, WSC '97.

[2]  Barry L. Nelson,et al.  Two-Stage Multiple Comparisons with the Best for Computer Simulation , 1995, Oper. Res..

[3]  S. Dalal,et al.  ALLOCATION OF OBSERVATIONS IN RANKING AND SELECTION WITH UNEQUAL VARIANCES , 1971 .

[4]  Shanti S. Gupta,et al.  Multiple decision procedures - theory and methodology of selecting and ranking populations , 1979, Classics in applied mathematics.

[5]  Barry L. Nelson,et al.  Ranking, selection and multiple comparisons in computer simulation , 1994, Proceedings of Winter Simulation Conference.

[6]  W. Chiu.  The ranking of means of normal populations for a generalized selection goal , 1974 .

[7]  Chun-Hung Chen,et al.  A gradient approach for smartly allocating computing budget for discrete event simulation , 1996, Winter Simulation Conference.

[8]  D. Berry.  Statistical Decision Theory, Foundations, Concepts, and Methods , 1982 .

[9]  John Cairns,et al.  The Economic Design of Sporting Contests , 2003 .

[10]  L. Moser,et al.  The Theory of Round Robin Tournaments , 1966 .

[11]  S. Gupta,et al.  Bayesian look ahead one-stage sampling allocations for selection of the best population , 1996 .

[12]  Barry L. Nelson,et al.  Comparing Systems via Simulation , 2007 .

[13]  Stephen E. Chick,et al.  New Procedures to Select the Best Simulated System Using Common Random Numbers , 2001, Manag. Sci..

[14]  S. Chick.  Selecting The Best System: A Decision-theoretic Approach , 1997, Winter Simulation Conference Proceedings,.

[15]  Y. Rinott.  On two-stage selection procedures and related probability-inequalities , 1978 .

[16]  Barry L. Nelson,et al.  Simultaneous Ranking, Selection and Multiple Comparisons for Simulation , 1993, Proceedings of 1993 Winter Simulation Conference - (WSC '93).

[17]  S. Gupta,et al.  Statistical decision theory and related topics IV , 1988 .

[18]  Chun-Hung Chen.  A lower bound for the correct subset-selection probability and its application to discrete-event system simulations , 1996, IEEE Trans. Autom. Control..

[19]  Peter R. Nelson,et al.  Multiple Comparisons: Theory and Methods , 1997 .

[20]  Chun-Hung Chen,et al.  An effective approach to smartly allocate computing budget for discrete event simulation , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.