Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations

Kriging metamodels (also called Gaussian process or spatial correlation models) approximate the Input/Output functions implied by the underlying simulation models. Such metamodels serve sensitivity analysis, especially for computationally expensive simulations. In practice, simulation analysts often know that this Input/Output function is monotonic. To obtain a Kriging metamodel that preserves this characteristic, this article uses distribution-free bootstrapping assuming each input combination is simulated several times to obtain more reliable averaged outputs. Nevertheless, these averages still show sampling variation, so the Kriging metamodel does not need to be an exact interpolator; bootstrapping gives a noninterpolating Kriging metamodel. Bootstrapping may use standard Kriging software. The method is illustrated through the popular M/M/1 model with either the mean or the 90% quantile as output; these outputs are monotonic functions of the traffic rate. The empirical results demonstrate that monotonicity-preserving bootstrapped Kriging gives higher probability of covering the true outputs, without lengthening the confidence interval.

[1]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox, Version 2.0 , 2002 .

[2]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[3]  Warren B. Powell,et al.  The Knowledge-Gradient Policy for Correlated Normal Beliefs , 2009, INFORMS J. Comput..

[4]  E. Jack Chen,et al.  Some Procedures of Selecting the Best Designs with Respect to Quantile , 2008, Simul..

[5]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[6]  Jack P. C. Kleijnen Design and Analysis of Simulation Experiments , 2007 .

[7]  Jack P. C. Kleijnen,et al.  Factor screening in simulation experiments : Review of sequential bifurcation , 2009 .

[8]  Jack P. C. Kleijnen,et al.  Constrained Optimization in Simulation: A Novel Approach , 2008 .

[9]  Marina Velikova,et al.  Monotone models for prediction in data mining , 2006 .

[10]  Ren-Jye Yang,et al.  Approximation methods in multidisciplinary analysis and optimization: a panel discussion , 2004 .

[11]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[12]  Agus Sudjianto,et al.  Blind Kriging: A New Method for Developing Metamodels , 2008 .

[13]  Walter J. Karplus,et al.  The Spectrum of Mathematical Models. , 1983 .

[14]  Xi Chen,et al.  The effects of common random numbers on stochastic kriging metamodels , 2012, TOMC.

[15]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[16]  A. Y. D. Siem,et al.  Property preservation and quality measures in meta-models , 2008 .

[17]  Jack P. C. Kleijnen,et al.  Kriging for interpolation in random simulation , 2003, J. Oper. Res. Soc..

[18]  A. J. Feelders Prior Knowledge in Economic Applications of Data Mining , 2000, PKDD.

[19]  Jack P. C. Kleijnen,et al.  Application-driven sequential designs for simulation experiments: Kriging metamodelling , 2004, J. Oper. Res. Soc..

[20]  Carl M. Harris,et al.  Fundamentals of queueing theory , 1975 .

[21]  F. Fred Choobineh,et al.  A quantile-based approach to system selection , 2010, Eur. J. Oper. Res..

[22]  Szu Hui Ng,et al.  A study on the effects of parameter estimation on kriging model's prediction error in stochastic simulations , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[23]  H. Dette,et al.  A simple nonparametric estimator of a strictly monotone regression function , 2006 .

[24]  Noel A Cressie,et al.  Statistics for Spatial Data, Revised Edition. , 1994 .

[25]  Michael C. Fu,et al.  Probabilistic Error Bounds for Simulation Quantile Estimators , 2003, Manag. Sci..

[26]  Jack P. C. Kleijnen,et al.  Constrained optimization in expensive simulation: Novel approach , 2010, Eur. J. Oper. Res..

[27]  Peter W. Glynn,et al.  Simulation-Based Response Surface Computation in the Presence of Monotonicity , 2006, Proceedings of the 2006 Winter Simulation Conference.

[28]  L. Jeff Hong,et al.  Estimating Quantile Sensitivities , 2009, Oper. Res..

[29]  Barry L. Nelson,et al.  Stochastic kriging for simulation metamodeling , 2008, 2008 Winter Simulation Conference.

[30]  Jack P. C. Kleijnen,et al.  The correct Kriging variance estimated by bootstrapping , 2006, J. Oper. Res. Soc..

[31]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[32]  Jerome Sacks,et al.  Choosing the Sample Size of a Computer Experiment: A Practical Guide , 2009, Technometrics.

[33]  A. B. Antognini,et al.  Exact optimal designs for computer experiments via Kriging metamodelling , 2010 .

[34]  Etienne de Klerk,et al.  Discrete least-norm approximation by nonnegative (trigonometric) polynomials and rational functions , 2005 .

[35]  E. Lehmann Elements of large-sample theory , 1998 .