Enhancing the Noise Robustness of the Optimal Computing Budget Allocation Approach

Since an optimal computing budget allocation (OCBA) approach maximizes the efficiency of the simulation budget allocation to correctly find the optimal solutions, various OCBA-based procedures, such as OCBA, OCBAm+, and MOCBA+, have been widely applied to solve simulation-based optimization problems. Recently, it has been found that the stochastic noise in a simulation model increases due to the increasing complexity of modern industrial systems. However, the OCBA approach may be inefficient for these practical problems. That is, it is very likely to waste a lot of budget on other candidates that are not truly optimal due to the abnormal simulation results, which occurs frequently in noisy environments. In this paper, we intuitively analyze the causes of this efficiency deterioration of the OCBA approach, and then a simple heuristic adjustment is proposed to enhance the noise robustness of the OCBA approach based on our analysis results. The proposed adjustment allows the OCBA approach to further consider the precision of the simulation results, thereby significantly reducing the wasted budget and increasing the efficiently. In addition, it can be applied to the existing allocation rules without modification and does not require additional computational costs. Many experimental results for the eight OCBA-based procedures clearly demonstrate the effectiveness of this adjustment. In particular, the results of practical problems emphasize its necessity.

[1]  Huashuai Qu,et al.  Simulation optimization: A tutorial overview and recent developments in gradient-based methods , 2014, Proceedings of the Winter Simulation Conference 2014.

[2]  Loo Hay Lee,et al.  Optimal Computing Budget Allocation to Select the Nondominated Systems—A Large Deviations Perspective , 2018, IEEE Transactions on Automatic Control.

[3]  Marina Schmid,et al.  Design And Analysis Of Experiments For Statistical Selection Screening And Multiple Comparisons , 2016 .

[4]  Barry L. Nelson,et al.  Recent advances in ranking and selection , 2007, 2007 Winter Simulation Conference.

[5]  Loo Hay Lee,et al.  A Simulation Budget Allocation Procedure for Enhancing the Efficiency of Optimal Subset Selection , 2016, IEEE Transactions on Automatic Control.

[6]  Loo Hay Lee,et al.  Optimal computing budget allocation for complete ranking with input uncertainty , 2020 .

[7]  MengChu Zhou,et al.  Approximately Optimal Computing Budget Allocation for Selection of the Best and Worst Designs , 2017, IEEE Transactions on Automatic Control.

[8]  Loo Hay Lee,et al.  Approximate Simulation Budget Allocation for Selecting the Best Design in the Presence of Stochastic Constraints , 2012, IEEE Transactions on Automatic Control.

[9]  Tag Gon Kim,et al.  Machine learning-based discrete event dynamic surrogate model of communication systems for simulating the command, control, and communication system of systems , 2019, Simul..

[10]  Barry L. Nelson,et al.  A fully sequential procedure for indifference-zone selection in simulation , 2001, TOMC.

[11]  Xiaoyu Lu,et al.  Hybrid Petri nets for modeling and analysis of microgrid systems , 2016, IEEE/CAA Journal of Automatica Sinica.

[12]  Tag Gon Kim,et al.  Efficient Ranking and Selection for Stochastic Simulation Model Based on Hypothesis Test , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Chun-Hung Chen,et al.  Efficient Simulation-Based Composition of Scheduling Policies by Integrating Ordinal Optimization With Design of Experiment , 2007, IEEE Transactions on Automation Science and Engineering.

[14]  Hui Xiao,et al.  Simulation budget allocation for simultaneously selecting the best and worst subsets , 2017, Autom..

[15]  Chun-Hung Chen,et al.  Opportunity Cost and OCBA Selection Procedures in Ordinal Optimization for a Fixed Number of Alternative Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[16]  Loo Hay Lee,et al.  Advances in simulation optimization and its applications , 2013 .

[17]  Hui Xiao,et al.  Optimal Budget Allocation Rule for Simulation Optimization Using Quadratic Regression in Partitioned Domains , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[18]  Hui Xiao,et al.  Optimal Computing Budget Allocation for Complete Ranking , 2014, IEEE Transactions on Automation Science and Engineering.

[19]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[20]  Loo Hay Lee,et al.  Computing budget allocation rules for multi-objective simulation models based on different measures of selection quality , 2010, Autom..

[21]  Loo Hay Lee,et al.  Discrete event simulation model for airline operations: SIMAIR , 2003, Proceedings of the 2003 Winter Simulation Conference, 2003..

[22]  Weiwei Chen,et al.  Efficient subset selection for the expected opportunity cost , 2015, Autom..

[23]  MengChu Zhou,et al.  Approximate Simulation Budget Allocation for Subset Ranking , 2017, IEEE Transactions on Control Systems Technology.

[24]  Jürgen Branke,et al.  Selecting a Selection Procedure , 2007, Manag. Sci..

[25]  Kyung-Min Seo,et al.  Measurement of Effectiveness for an Anti-torpedo Combat System Using a Discrete Event Systems Specification-based Underwater Warfare Simulator , 2011 .

[26]  Hui Xiao,et al.  An efficient simulation procedure for ranking the top simulated designs in the presence of stochastic constraints , 2019, Autom..

[27]  Peter W. Glynn,et al.  A large deviations perspective on ordinal optimization , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[28]  Loo Hay Lee,et al.  Simulation optimization using the cross-entropy method with optimal computing budget allocation , 2010, TOMC.

[29]  Alexander M. Wyglinski,et al.  Vehicular Network Simulation Environment via Discrete Event System Modeling , 2019, IEEE Access.

[30]  Jiacun Wang,et al.  A simulation engine for stochastic timed petri nets and application to emergency healthcare systems , 2019, IEEE/CAA Journal of Automatica Sinica.

[31]  Leyuan Shi,et al.  A New Budget Allocation Framework for the Expected Opportunity Cost , 2017, Oper. Res..

[32]  Loo Hay Lee,et al.  Stochastically Constrained Ranking and Selection via SCORE , 2014, ACM Trans. Model. Comput. Simul..

[33]  Loo Hay Lee,et al.  Optimal Computing Budget Allocation for Particle Swarm Optimization in Stochastic Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[34]  Loo Hay Lee,et al.  Simulation Optimization: A Review and Exploration in the New Era of Cloud Computing and Big Data , 2015, Asia Pac. J. Oper. Res..

[35]  Tag Gon Kim,et al.  Pareto Set Selection for Multiobjective Stochastic Simulation Model , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[36]  Loo Hay Lee,et al.  Multi-objective simulation-based evolutionary algorithm for an aircraft spare parts allocation problem , 2008, Eur. J. Oper. Res..

[37]  L. Lee,et al.  Finding the non-dominated Pareto set for multi-objective simulation models , 2010 .

[38]  Tag Gon Kim,et al.  Model-Based Design of Defense Cyber-Physical Systems to Analyze Mission Effectiveness and Network Performance , 2019, IEEE Access.

[39]  Loo Hay Lee,et al.  Efficient Simulation Budget Allocation for Selecting an Optimal Subset , 2008, INFORMS J. Comput..

[40]  John F. Shortle,et al.  Optimal splitting for rare-event simulation , 2012 .

[41]  Hui Xiao,et al.  Simulation Budget Allocation for Selecting the Top-m Designs With Input Uncertainty , 2018, IEEE Transactions on Automatic Control.

[42]  MengChu Zhou,et al.  Incorporation of Optimal Computing Budget Allocation for Ordinal Optimization Into Learning Automata , 2016, IEEE Transactions on Automation Science and Engineering.

[43]  Stephen E. Chick,et al.  New Procedures to Select the Best Simulated System Using Common Random Numbers , 2001, Manag. Sci..

[44]  Tag Gon Kim,et al.  A Heuristic Procedure for Selecting the Best Feasible Design in the Presence of Stochastic Constraints , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[45]  Shih-Cheng Horng,et al.  Applying PSO and OCBA to Minimize the Overkills and Re-Probes in Wafer Probe Testing , 2012, IEEE Transactions on Semiconductor Manufacturing.

[46]  Tag Gon Kim,et al.  An Improved Budget Allocation Procedure for Selecting the Best-Simulated Design in the Presence of Large Stochastic Noise , 2019, IEEE Access.

[47]  Guangli Zhu,et al.  Stochastic Process and Simulation of Traction Load for High Speed Railways , 2019, IEEE Access.

[48]  R. H. Smith Optimization for Simulation : Theory vs . Practice , 2002 .

[49]  Zeshui Xu,et al.  A Bilevel Model for Project Scheduling in a Fuzzy Random Environment , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[50]  Tag Gon Kim,et al.  A Heuristic Approach for Selecting Best-Subset Including Ranking Within the Subset , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[51]  Jingde Cheng,et al.  Adaptive Evaluation of Virtual Machine Placement and Migration Scheduling Algorithms Using Stochastic Petri Nets , 2019, IEEE Access.

[52]  Tag Gon Kim,et al.  Optical engineering simulator development for efficient design of miles gear , 2015 .

[53]  Y. Ho,et al.  Vector Ordinal Optimization , 2005 .

[54]  Qing-Shan Jia,et al.  Efficient Computing Budget Allocation for Simulation-Based Policy Improvement , 2012, IEEE Transactions on Automation Science and Engineering.

[55]  Jie Song,et al.  Integrating Optimal Simulation Budget Allocation and Genetic Algorithm to Find the Approximate Pareto Patient Flow Distribution , 2016, IEEE Transactions on Automation Science and Engineering.

[56]  Chun-Hung Chen,et al.  Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization , 2000, Discret. Event Dyn. Syst..