Optimal Computing Budget Allocation for Complete Ranking

Previous research in ranking and selection focused on selecting the best design and subset selection. Little research has been done for ranking all designs completely. Complete ranking has been applied to design of experiment, random number generator and population-based search algorithms. In this paper, we consider the problem of ranking all designs. Our objective is to develop an efficient simulation allocation procedure that maximizes the probability of correct ranking with fixed limited computing budget. A previous allocation strategy of complete ranking based on indifference zone formulation is conservative and not efficient enough. We use the optimal computing budget allocation framework to further enhance the efficiency and reduce the amount of budget needed to achieve the same probability of correct ranking. Compared with the previous allocation strategy, our proposed allocation rule performs best under different scenarios.

[1]  Loo Hay Lee,et al.  Multi-objective simulation-based evolutionary algorithm for an aircraft spare parts allocation problem , 2008, Eur. J. Oper. Res..

[2]  Thomas A. Bishop.  Designing simulation experiments to completely rank alternatives , 1978, WSC '78.

[3]  Damon Wischik,et al.  Big queues , 2004, Lecture notes in mathematics.

[4]  Susan R. Hunter,et al.  Large-deviation sampling laws for constrained simulation optimization on finite sets , 2010, Proceedings of the 2010 Winter Simulation Conference.

[5]  Jürgen Branke,et al.  Selecting a Selection Procedure , 2007, Manag. Sci..

[6]  Chun-Hung Chen,et al.  Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization , 2000, Discret. Event Dyn. Syst..

[7]  L. Lee,et al.  Finding the non-dominated Pareto set for multi-objective simulation models , 2010 .

[8]  Barry L. Nelson,et al.  Chapter 17 Selecting the Best System , 2006, Simulation.

[9]  Jürgen Branke,et al.  Integrating Techniques from Statistical Ranking into Evolutionary Algorithms , 2006, EvoWorkshops.

[10]  Chun-Hung Chen,et al.  Opportunity Cost and OCBA Selection Procedures in Ordinal Optimization for a Fixed Number of Alternative Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[11]  Loo Hay Lee,et al.  Efficient Simulation Budget Allocation for Selecting an Optimal Subset , 2008, INFORMS J. Comput..

[12]  Loo Hay Lee,et al.  Simulation optimization using the cross-entropy method with optimal computing budget allocation , 2010, TOMC.

[13]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[14]  Loo Hay Lee,et al.  Computing budget allocation rules for multi-objective simulation models based on different measures of selection quality , 2010, Autom..

[15]  Edward J. Dudewicz,et al.  Complete statistical ranking of populations, with tables and applications☆ , 1982 .

[17]  Loo Hay Lee,et al.  Optimal computing budget allocation for multi-objective simulation models , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[18]  Peter W. Glynn,et al.  A large deviations perspective on ordinal optimization , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[19]  Loo Hay Lee,et al.  Optimal Computing Budget Allocation for constrained optimization , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[20]  Lothar Thiele,et al.  A Comparison of Selection Schemes used in Genetic Algorithms , 1995 .

[21]  Shih-Cheng Horng,et al.  Ordinal optimization based approach to the optimal resource allocation of grid computing system , 2011, Math. Comput. Model..