Sequential design of experiments for estimating percentiles of black-box functions

Estimating percentiles of black-box deterministic functions with random inputs is a challenging task when the number of function evaluations is severely restricted, which is typical for computer experiments. This article proposes two new sequential Bayesian methods for percentile estimation based on the Gaussian Process metamodel. Both rely on the Stepwise Uncertainty Reduction paradigm, hence aim at providing a sequence of function evaluations that reduces an uncertainty measure associated with the percentile estimator. The proposed strategies are tested on several numerical examples, showing that accurate estimators can be obtained using only a small number of functions evaluations.

[1]  Jean-Marie Cornuet,et al.  Adaptive Multiple Importance Sampling , 2009, 0907.1254.

[2]  Warren B. Powell,et al.  The Correlated Knowledge Gradient for Simulation Optimization of Continuous Parameters using Gaussian Process Regression , 2011, SIAM J. Optim..

[3]  Ling Li,et al.  Sequential design of computer experiments for the estimation of a probability of failure , 2010, Statistics and Computing.

[4]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[5]  G. M. Tallis.  The Moment Generating Function of the Truncated Multi‐Normal Distribution , 1961 .

[6]  Mathieu Couplet,et al.  Évaluation d'un risque d'inondation fluviale par planification séquentielle d'expériences , 2010 .

[7]  Donald Geman,et al.  An Active Testing Model for Tracking Roads in Satellite Images , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Yves Deville,et al.  DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization , 2012 .

[9]  Eric Moulines,et al.  Sequential Design of Computer Experiments for the Assessment of Fetus Exposure to Electromagnetic Fields , 2016, Technometrics.

[10]  Victor Picheny,et al.  Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reduction , 2013, Statistics and Computing.

[11]  Alexander J. Smola,et al.  Exponential Regret Bounds for Gaussian Process Bandits with Deterministic Observations , 2012, ICML.

[12]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[13]  Daniel Egloff,et al.  QUANTILE ESTIMATION WITH ADAPTIVE IMPORTANCE SAMPLING , 2010, 1002.4946.

[14]  Ping Zhang.  Nonparametric Importance Sampling , 1996 .

[15]  Jérôme Morio,et al.  Extreme quantile estimation with nonparametric adaptive importance sampling , 2012, Simul. Model. Pract. Theory.

[16]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[17]  Jonas Mockus,et al.  On Bayesian Methods for Seeking the Extremum , 1974, Optimization Techniques.

[18]  Victor Picheny,et al.  Fast Parallel Kriging-Based Stepwise Uncertainty Reduction With Application to the Identification of an Excursion Set , 2014, Technometrics.

[19]  B. Iooss,et al.  Controlled stratification for quantile estimation , 2008, 0802.2426.

[20]  Robert B. Gramacy,et al.  Ja n 20 08 Bayesian Treed Gaussian Process Models with an Application to Computer Modeling , 2009 .

[21]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[22]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[23]  David Ginsbourger,et al.  Corrected Kriging Update Formulae for Batch-Sequential Data Assimilation , 2012, 1203.6452.

[24]  J. Oakley.  Estimating percentiles of uncertain computer code outputs , 2004 .

[25]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[26]  Eric Moulines,et al.  Sequential design of computer experiments for parameter estimation with application to numerical dosimetry , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[27]  Maurice Lemaire,et al.  Assessing small failure probabilities by combined subset simulation and Support Vector Machines , 2011 .

[28]  Eric Walter,et al.  An informational approach to the global optimization of expensive-to-evaluate functions , 2006, J. Glob. Optim..