Kriging metamodels and global opimization in simulation

Simulation is a popular tool for analyzing complex systems. However, simulation models are often difficult to build and require significant time to run. We often need to invest much money and time to use a simulation model of a complex system. To benefit more from a simulation investment, we may use the simulation input/output data to build a metamodel (model of the simulation model). This metamodel may adequately approximate the original simulation model, but is simpler to build and run. Though there are many types of metamodels, we focus on Kriging (or Gaussian process) models. The advantage of Kriging is that it not only predicts the simulation output but also quantifies the prediction uncertainty. We may use the Kriging metamodel to guide the search for the global optimum of the simulation model. A popular method for the global optimization of deterministic simulation is so-called efficient global optimization (EGO) and its expected improvement (EI). EGO uses a Kriging metamodel to calculate EI, and balances exploitation (local search) and exploration (global search) when searching for the optimal input combination for the simulation model. In this dissertation, we investigate several methodological questions about Kriging metamodels and their use in EGO for deterministic and random simulation models. In Chapter 2, we study multivariate Kriging versus univariate Kriging for simulation models with multiple responses. In Chapter 3, we focus on two related questions: (1) How to select the next combination to be simulated when searching for the global optimum? (2) How to derive confidence intervals for outputs of input combinations not yet simulated? In Chapter 4, we study Kriging metamodels that are required to be either convex or monotonic. In Chapter 5, we introduce intrinsic Kriging as a metamodel of deterministic and random simulation models. In Chapter 6, we study the use of intrinsic Kriging as a new metamodel in global optimization of deterministic and random simulations.

[1]  M. Wand,et al.  Kriging with Nonparametric Variance Function Estimation , 1999, Biometrics.

[2]  H. Wackernagle,et al.  Multivariate geostatistics: an introduction with applications , 1998 .

[3]  David Ginsbourger,et al.  Quantifying uncertainty on Pareto fronts with Gaussian process conditional simulations , 2015, Eur. J. Oper. Res..

[4]  Bertrand Iooss,et al.  Global sensitivity analysis of stochastic computer models with joint metamodels , 2008, Statistics and Computing.

[5]  Jerome Sacks,et al.  Choosing the Sample Size of a Computer Experiment: A Practical Guide , 2009, Technometrics.

[6]  Victor Picheny,et al.  Quantile-Based Optimization of Noisy Computer Experiments With Tunable Precision , 2013, Technometrics.

[7]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[8]  Jack P. C. Kleijnen,et al.  Conditional simulation for efficient global optimization , 2013, 2013 Winter Simulations Conference (WSC).

[9]  W. Whitt.  Planning queueing simulations , 1989 .

[10]  Alan J. Mayne,et al.  Towards Global Optimisation 2 , 1976 .

[11]  Robert B. Gramacy,et al.  Cases for the nugget in modeling computer experiments , 2010, Statistics and Computing.

[12]  David Ginsbourger,et al.  Multiples métamodèles pour l'approximation et l'optimisation de fonctions numériques multivariables , 2009 .

[13]  Barry L. Nelson,et al.  Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening , 2010, INFORMS J. Comput..

[14]  Jack P. C. Kleijnen,et al.  Stochastic Intrinsic Kriging for Simulation Metamodelling , 2015 .

[15]  Karin S. Dorman,et al.  mlegp: statistical analysis for computer models of biological systems using R , 2008, Bioinform..

[16]  Eric Walter,et al.  Intrinsic Kriging and prior information , 2005 .

[17]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[18]  Jack P. C. Kleijnen,et al.  Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations , 2009, J. Oper. Res. Soc..

[19]  J. Chilès,et al.  Geostatistics: Modeling Spatial Uncertainty , 1999 .

[20]  Robert E. Shannon,et al.  Design and analysis of simulation experiments , 1978, WSC '78.

[21]  Christine A. Shoemaker,et al.  Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems , 2014, J. Glob. Optim..

[22]  Tom Dhaene,et al.  Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization , 2014, J. Glob. Optim..

[23]  Yves Deville,et al.  DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization , 2012 .

[24]  Warren B. Powell,et al.  Calibrating simulation models using the knowledge gradient with continuous parameters , 2010, Proceedings of the 2010 Winter Simulation Conference.

[25]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[26]  Jack P. C. Kleijnen,et al.  Constrained Optimization in Simulation: A Novel Approach , 2008 .

[27]  Barry L. Nelson,et al.  Stochastic kriging for simulation metamodeling , 2008, 2008 Winter Simulation Conference.

[28]  Jack P. C. Kleijnen,et al.  Kriging Metamodeling in Simulation: A Review , 2007, Eur. J. Oper. Res..

[29]  Sébastien Da Veiga,et al.  Gaussian process modeling with inequality constraints , 2012 .

[30]  Chun-Hung Chen,et al.  Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization , 2000, Discret. Event Dyn. Syst..

[31]  D. F. Marks,et al.  An introduction , 1988, Experientia.

[32]  Layne T. Watson,et al.  Efficient global optimization algorithm assisted by multiple surrogate techniques , 2012, Journal of Global Optimization.

[33]  Xi Chen,et al.  Common random numbers and stochastic kriging , 2010, Proceedings of the 2010 Winter Simulation Conference.

[34]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[35]  Jack P. C. Kleijnen,et al.  The correct Kriging variance estimated by bootstrapping , 2006, J. Oper. Res. Soc..

[36]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[37]  Raphael T. Haftka,et al.  Performance Estimate and Simultaneous Application of Multiple Surrogates , 2006 .

[38]  Nicolas Gayton,et al.  AK-MCS: An active learning reliability method combining Kriging and Monte Carlo Simulation , 2011 .

[39]  Loo Hay Lee,et al.  Simulation optimization via kriging: a sequential search using expected improvement with computing budget constraints , 2013 .

[40]  Robert Haining,et al.  Statistics for spatial data: by Noel Cressie, 1991, John Wiley & Sons, New York, 900 p., ISBN 0-471-84336-9, US $89.95 , 1993 .

[41]  Izzet Sahin,et al.  On the Objective Function Behavior in (s, S) Inventory Models , 1982, Oper. Res..

[42]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[43]  N. Zheng,et al.  Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models , 2006, J. Glob. Optim..

[44]  Runze Li,et al.  Analysis of Computer Experiments Using Penalized Likelihood in Gaussian Kriging Models , 2005, Technometrics.

[45]  S. Azarm,et al.  Approximation of multiresponse deterministic engineering simulations: a dependent metamodeling approach , 2006 .

[46]  Jack P. C. Kleijnen,et al.  Techniques for sensitivity analysis of simulation models: A case study of the CO2 greenhouse effect , 1992, Simul..

[47]  Herbert K. H. Lee,et al.  Bayesian Guided Pattern Search for Robust Local Optimization , 2009, Technometrics.

[48]  Wei Xie,et al.  The influence of correlation functions on stochastic kriging metamodels , 2010, Proceedings of the 2010 Winter Simulation Conference.

[49]  C. J. Lawrence.  Robust estimates of location : survey and advances , 1975 .

[50]  Szu Hui Ng,et al.  A Bayesian metamodeling approach for stochastic simulations , 2010, Proceedings of the 2010 Winter Simulation Conference.

[51]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[52]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox, Version 2.0 , 2002 .

[53]  Stéphanie Mahévas,et al.  ISIS-Fish, a generic and spatially explicit simulation tool for evaluating the impact of management measures on fisheries dynamics , 2004 .

[54]  Jack P. C. Kleijnen,et al.  Simulation and optimization in production planning: A case study , 1993, Decis. Support Syst..

[55]  G. Matheron.  The intrinsic random functions and their applications , 1973, Advances in Applied Probability.

[56]  Jack P. C. Kleijnen,et al.  Performance metrics in supply chain management , 2003, J. Oper. Res. Soc..

[57]  Donald R. Jones,et al.  Global optimization of deceptive functions with sparse sampling , 2008 .

[58]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .