Random Perturbation and Bagging to Quantify Input Uncertainty

We consider the problem of estimating the output variance in simulation analysis that is contributed from the statistical errors in fitting the input models, the latter often known as the input uncertainty. This variance contribution can be written in terms of the sensitivity estimate of the output and the variance of the input distributions or parameters, via the delta method. We study the direct use of this representation in obtaining efficient estimators for the input-contributed variance, by using finite-difference and random perturbation to approximate the gradient, focusing especially in the nonparametric case. In particular, we analyze a particular type of random perturbation motivated from resampling that connects to an infinitesimal jackknife estimator used in bagging. We illustrate the optimal simulation allocation and the simulation effort complexity of this scheme, and show some supporting numerical results.

[1]  R. Cheng,et al.  Two-point methods for assessing variability in simulation output , 1998 .

[2]  Faker Zouaoui,et al.  Accounting for input-model and input-parameter uncertainties in simulation , 2004 .

[3]  Yurii Nesterov,et al.  Random Gradient-Free Minimization of Convex Functions , 2015, Foundations of Computational Mathematics.

[4]  R. R. Barton,et al.  Tutorial: Input uncertainty in outout analysis , 2012, Proceedings Title: Proceedings of the 2012 Winter Simulation Conference (WSC).

[5]  James R. Wilson,et al.  Accounting for parameter uncertainty in simulation input modeling , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[6]  Barry L. Nelson,et al.  Single-experiment input uncertainty , 2015, J. Simulation.

[7]  James R. Wilson,et al.  Accounting for Parameter Uncertainty in Simulation Input Modeling , 2003 .

[8]  Stephen E. Chick,et al.  Bayesian Ideas and Discrete Event Simulation: Why, What and How , 2006, Proceedings of the 2006 Winter Simulation Conference.

[9]  Barry L. Nelson,et al.  Foundations and Methods of Stochastic Simulation: A First Course , 2013 .

[10]  Barry L. Nelson,et al.  Advanced tutorial: Input uncertainty quantification , 2014, Proceedings of the Winter Simulation Conference 2014.

[11]  Barry L. Nelson,et al.  Input-Output Uncertainty Comparisons for Discrete Optimization via Simulation , 2019, Oper. Res..

[12]  Stefun D. Leigh.  U-Statistics Theory and Practice , 1992 .

[13]  F. Hampel.  The Influence Curve and Its Role in Robust Estimation , 1974 .

[14]  Bruce W. Schmeiser,et al.  Stochastic Gradient Estimation using a Single Design Point , 2006, Proceedings of the 2006 Winter Simulation Conference.

[15]  R. Cheng,et al.  Sensitivity of computer simulation experiments to errors in input data , 1997 .

[16]  P. Bühlmann,et al.  Analyzing Bagging , 2001 .

[17]  Shane G. Henderson,et al.  Input modeling: input model uncertainty: why do we care and what should we do about it? , 2003, WSC '03.

[18]  B. Efron.  Estimation and Accuracy After Model Selection , 2014, Journal of the American Statistical Association.

[19]  Barry L. Nelson,et al.  Quickly Assessing Contributions to Input Uncertainty , 2015 .

[20]  A. Skoogh,et al.  CONSTRUCTING SIMULATION OUTPUT INTERVALS UNDER INPUT UNCERTAINTY VIA DATA SECTIONING , 2018 .

[21]  P. Frazier,et al.  Advanced tutorial: Input uncertainty and robust analysis in stochastic simulation , 2016, 2016 Winter Simulation Conference (WSC).

[22]  Adam Tauman Kalai,et al.  Online convex optimization in the bandit setting: gradient descent without a gradient , 2004, SODA '05.

[23]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.