A Review of Optimal Computing Budget Allocation Algorithms for Simulation Optimization Problem

 Simulation and optimization are two arguably most used operations research (OR) tools. Optimization intends to choose the best element from some set of available alternatives. Stochastic simulation is a powerful modeling and software tool for analyzing modern complex systems. This capability complements the inherent limitation of traditional optimization, so the combining use of simulation and optimization is growing in popularity. While the advance of new technology has dramatically increased computational power, efficiency is still a big concern because many simulation replications are required for each performance evaluation. Optimal Computing Budget Allocation (OCBA) algorithms have been developed to address such an efficiency issue with emphasis given on those aiming to maximize the probability of correct selection or other measures of selection quality given a limited computing budget. In this paper, we present a comprehensive survey on OCBA approaches for various simulation optimization problems together with the open challenges for future research. KeywordsOptimization; Discrete-event simulation; Simulation optimization; Ranking and selection; Computing budget allocation.

[1]  S. Dalal,et al.  ALLOCATION OF OBSERVATIONS IN RANKING AND SELECTION WITH UNEQUAL VARIANCES , 1971 .

[2]  Y. Rinott.  On two-stage selection procedures and related probability-inequalities , 1978 .

[3]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[4]  L. Dai.  Convergence properties of ordinal comparison in the simulation of discrete event dynamic systems , 1995 .

[5]  Chun-Hung Chen,et al.  An effective approach to smartly allocate computing budget for discrete event simulation , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[6]  Chun-Hung Chen,et al.  A gradient approach for smartly allocating computing budget for discrete event simulation , 1996, Winter Simulation Conference.

[7]  Chun-Hung Chen.  A lower bound for the correct subset-selection probability and its application to discrete-event system simulations , 1996, IEEE Trans. Autom. Control..

[8]  Chun-Hung Chen,et al.  New development of optimal computing budget allocation for discrete event simulation , 1997, WSC '97.

[9]  Chun-Hung Chen,et al.  Computing budget allocation for simulation experiments with different system structures , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[10]  Chun-Hung Chen,et al.  An asymptotic allocation for simultaneous simulation experiments , 1999, WSC '99.

[11]  Loo Hay Lee,et al.  Explanation of goal softening in ordinal optimization , 1999, IEEE Trans. Autom. Control..

[12]  Chun-Hung Chen,et al.  Ordinal comparison of heuristic algorithms using stochastic optimization , 1999, IEEE Trans. Robotics Autom..

[13]  Chun-Hung Chen,et al.  An asymptotic allocation for simultaneous simulation experiments , 1999, WSC'99. 1999 Winter Simulation Conference Proceedings. 'Simulation - A Bridge to the Future' (Cat. No.99CH37038).

[14]  Chun-Hung Chen,et al.  An empirical evaluation of several methods to select the best system , 1999, TOMC.

[15]  Leyuan Shi,et al.  Simultaneous simulation experiments and nested partition for discrete resource allocation in supply chain management , 1999, WSC '99.

[16]  Chun-Hung Chen,et al.  Convergence Properties of Two-Stage Stochastic Programming , 2000 .

[17]  Leyuan Shi.  A New Algorithm for Stochastic Discrete Resource Allocation Optimization , 2000, Discret. Event Dyn. Syst..

[18]  Chun-Hung Chen,et al.  Computing efforts allocation for ordinal optimization and discrete event simulation , 2000, IEEE Trans. Autom. Control..

[19]  Chun-Hung Chen,et al.  Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization , 2000, Discret. Event Dyn. Syst..

[20]  Barry L. Nelson,et al.  A fully sequential procedure for indifference-zone selection in simulation , 2001, TOMC.

[21]  Stephen E. Chick,et al.  New Procedures to Select the Best Simulated System Using Common Random Numbers , 2001, Manag. Sci..

[22]  Chun-Hung Chen,et al.  Scheduling semiconductor wafer fabrication by using ordinal optimization-based simulation , 2001, IEEE Trans. Robotics Autom..

[23]  M.C. Fu,et al.  Simulation optimization , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[24]  Julie L. Swann,et al.  Simple Procedures for Selecting the Best Simulated System When the Number of Alternatives is Large , 2001, Oper. Res..

[25]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[26]  R. H. Smith.  Optimization for Simulation : Theory vs . Practice , 2002 .

[27]  Ek Peng Chew,et al.  A simulation study on sampling and selecting under fixed computing budget , 2003, Proceedings of the 2003 Winter Simulation Conference, 2003..

[28]  P. Sánchez,et al.  SELECTING THE BEST SYSTEM : THEORY AND METHODS , 2003 .

[29]  Chun-Hung Chen,et al.  Optimal computing budget allocation for Monte Carlo simulation with application to product design , 2003, Simul. Model. Pract. Theory.

[30]  P. Sánchez,et al.  BETTER-THAN-OPTIMAL SIMULATION RUN ALLOCATION ? , 2003 .

[31]  Enver Yücesan,et al.  Discrete-event simulation optimization using ranking, selection, and multiple comparison procedures: A survey , 2003, TOMC.

[32]  Loo Hay Lee,et al.  Optimal computing budget allocation for multi-objective simulation models , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[33]  Peter W. Glynn,et al.  A large deviations perspective on ordinal optimization , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[34]  Lucy Y. Pao,et al.  Computing budget allocation for efficient ranking and selection of variances with application to target tracking algorithms , 2004, IEEE Transactions on Automatic Control.

[35]  Chun-Hung Chen,et al.  Optimal computing budget allocation under correlated sampling , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[36]  David M. Nicol,et al.  Proceedings of the 2004 Winter Simulation Conference , 2004 .

[37]  Ihsan Sabuncuoglu,et al.  Simulation optimization: A comprehensive review on theory and applications , 2004 .

[38]  Chun-Hung Chen,et al.  A moving mesh approach for simulation budget allocation on continuous domains , 2005, Proceedings of the Winter Simulation Conference, 2005..

[39]  Chun-Hung Chen,et al.  An alternative simulation budget allocation scheme for efficient simulation , 2005, Int. J. Simul. Process. Model..

[40]  Yaozhong Wu,et al.  Selection Procedures with Frequentist Expected Opportunity Cost Bounds , 2005, Oper. Res..

[41]  Chun-Hung Chen,et al.  Intelligent simulation for alternatives comparison and application to air traffic management , 2005 .

[42]  Analysis methodology: are we done? , 2005, WSC.

[43]  M. Kuhl,et al.  APPLICATION OF MULTI-OBJECTIVE SIMULATION-OPTIMIZATION TECHNIQUES TO INVENTORY MANAGEMENT PROBLEMS , 2005 .

[44]  Chun-Hung Chen,et al.  Demonstration of probabilistic ordinal optimization concepts for continuous-variable optimization under uncertainty , 2006 .

[45]  Loo Hay Lee,et al.  A new approach to discrete stochastic optimization problems , 2006, Eur. J. Oper. Res..

[46]  Chun-Hung Chen,et al.  Efficient Dynamic Simulation Allocation in Ordinal Optimization , 2006, IEEE Transactions on Automatic Control.

[47]  Loo Hay Lee,et al.  A general framework on the simulation-based optimization under fixed computing budget , 2006, Eur. J. Oper. Res..

[48]  Loo Hay Lee,et al.  Integration of Statistical Selection with Search Mechanism for Solving Multi-Objective Simulation-Optimization Problems , 2006, Proceedings of the 2006 Winter Simulation Conference.

[49]  Barry L. Nelson,et al.  Recent advances in ranking and selection , 2007, 2007 Winter Simulation Conference.

[50]  Barry L. Nelson,et al.  Comparing Systems via Simulation , 2007 .

[51]  Loo Hay Lee,et al.  Finding the pareto set for multi-objective simulation models by minimization of expected opportunity cost , 2007, 2007 Winter Simulation Conference.

[52]  Jürgen Branke,et al.  Selecting a Selection Procedure , 2007, Manag. Sci..

[53]  Chun-Hung Chen,et al.  Efficient Simulation-Based Composition of Scheduling Policies by Integrating Ordinal Optimization With Design of Experiment , 2007, IEEE Transactions on Automation Science and Engineering.

[54]  Jrgen Branke.  EFFICIENT SAMPLING IN INTERACTIVE MULTI-CRITERIA SELECTION , 2007 .

[55]  Loo Hay Lee,et al.  Multi-objective ordinal optimization for simulation optimization problems , 2007, Autom..

[56]  Chun-Hung Chen,et al.  Simulation Allocation for Determining the Best Design in the Presence of Correlated Sampling , 2007, INFORMS J. Comput..

[57]  Chun-Hung Chen,et al.  An efficient Ranking and Selection procedure for a linear transient mean performance measure , 2008, 2008 Winter Simulation Conference.

[58]  Enver Yücesan,et al.  A new perspective on feasibility determination , 2008, 2008 Winter Simulation Conference.

[59]  Loo Hay Lee,et al.  Multi-objective simulation-based evolutionary algorithm for an aircraft spare parts allocation problem , 2008, Eur. J. Oper. Res..

[60]  Jingchen Liu,et al.  Large deviations perspective on ordinal optimization of heavy-tailed systems , 2008, 2008 Winter Simulation Conference.

[61]  Loo Hay Lee,et al.  Efficient Simulation Budget Allocation for Selecting an Optimal Subset , 2008, INFORMS J. Comput..

[62]  Leyuan Shi,et al.  Some topics for simulation optimization , 2008, 2008 Winter Simulation Conference.

[63]  Loo Hay Lee,et al.  Optimal sampling in design of experiment for simulation-based stochastic optimization , 2008, 2008 IEEE International Conference on Automation Science and Engineering.

[64]  Warren B. Powell,et al.  The knowledge-gradient stopping rule for ranking and selection , 2008, 2008 Winter Simulation Conference.

[65]  Chun-Hung Chen,et al.  A preliminary study of optimal splitting for rare-event simulation , 2008, 2008 Winter Simulation Conference.

[66]  Loo Hay Lee,et al.  A multi-objective selection procedure of determining a Pareto set , 2009, Comput. Oper. Res..

[67]  Loo Hay Lee,et al.  Optimal Computing Budget Allocation for constrained optimization , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[68]  Barry L. Nelson,et al.  A brief introduction to optimization via simulation , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[69]  Chun-Hung Chen,et al.  A transient means ranking and selection procedure with sequential sampling constraints , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[70]  Loo Hay Lee,et al.  Differentiated service inventory optimization using nested partitions and MOCBA , 2009, Comput. Oper. Res..

[71]  L. Lee,et al.  Finding the non-dominated Pareto set for multi-objective simulation models , 2010 .

[72]  Jürgen Branke,et al.  Sequential Sampling to Myopically Maximize the Expected Value of Information , 2010, INFORMS J. Comput..

[73]  Loo Hay Lee,et al.  Integration of indifference-zone with multi-objective computing budget allocation , 2010, Eur. J. Oper. Res..

[74]  Loo Hay Lee,et al.  Stochastic Simulation Optimization - An Optimal Computing Budget Allocation , 2010, System Engineering and Operations Research.

[75]  F. Fred Choobineh,et al.  A quantile-based approach to system selection , 2010, Eur. J. Oper. Res..

[76]  Loo Hay Lee,et al.  Simulation optimization using the cross-entropy method with optimal computing budget allocation , 2010, TOMC.