Bayesian treed gaussian process models

Computer experiments often require dense sweeps over input parameters to obtain a qualitative understanding of their response. Such sweeps can be prohibitively expensive, and are unnecessary in regions where the response is easily predicted; well-chosen designs could allow a mapping of the response with far fewer simulation runs. Thus, there is a need for computationally inexpensive surrogate models and an accompanying method for selecting small designs. This dissertation explores a nonparametric and semiparametric nonstationary modeling methodologies for addressing this need that couples stationary Gaussian processes and (limiting) linear models with treed partitioning. A Bayesian perspective yields an explicit measure of (nonstationary) predictive uncertainty that can be used to guide sampling. As typical experiments are high-dimensional and require large designs, a careful but thrifty implementation is essential. The methodological developments and statistical computing details which make this approach efficient are outlined in detail. In addition to several illustrations using synthetic data, classic nonstationary data analyzed in recent literature are used to validate the model, and the benefit of adaptive sampling is illustrated through a motivating example which involves the computational fluid dynamics simulation of a NASA reentry vehicle.

[1]  B. Mallick,et al.  Analyzing Nonstationary Spatial Data Using Piecewise Gaussian Processes , 2005 .

[2]  Robert B. Gramacy,et al.  Parameter space exploration with Gaussian process trees , 2004, ICML.

[3]  Sw. Banerjee,et al.  Hierarchical Modeling and Analysis for Spatial Data , 2003 .

[4]  A. O'Hagan,et al.  Bayesian inference for non‐stationary spatial covariance structure via spatial deformations , 2003 .

[5]  T. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer series in statistics.

[6]  Edward I. George,et al.  Bayesian Treed Models , 2002, Machine Learning.

[7]  D. Hand,et al.  Bayesian partition modelling , 2002 .

[8]  J. Berger,et al.  Objective Bayesian Analysis of Spatially Correlated Data , 2001 .

[9]  Michael L. Stein,et al.  Interpolation of spatial data , 1999 .

[10]  Adrian F. M. Smith,et al.  A Bayesian CART algorithm , 1998 .

[11]  Eric R. Ziegel,et al.  Practical Nonparametric and Semiparametric Bayesian Statistics , 1998, Technometrics.

[12]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[13]  Radford M. Neal Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification , 1997, physics/9701026.

[14]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[15]  D. Dunson,et al.  Bayesian Data Analysis. , 1996 .

[16]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[17]  Noel A Cressie,et al.  Statistics for Spatial Data, Revised Edition. , 1994 .

[18]  P. Guttorp,et al.  Nonparametric Estimation of Nonstationary Spatial Covariance Structure , 1992 .

[19]  A. O'Hagan,et al.  Bayes–Hermite quadrature , 1991 .

[20]  J. Friedman Multivariate adaptive regression splines , 1990 .

[21]  D. Rubinfeld,et al.  Hedonic housing prices and the demand for clean air , 1978 .

[22]  G. Matheron Principles of geostatistics , 1963 .

[23]  Wei Chu,et al.  Bayesian support vector regression using a unified loss function , 2004, IEEE Transactions on Neural Networks.

[24]  D. Higdon Space and Space-Time Modeling using Process Convolutions , 2002 .

[25]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[26]  J. Besag,et al.  On Bayesian analysis of mixtures with an unknown number of components. Discussion. Author's reply , 1997 .

[27]  N. Hjort,et al.  Topics in Spatial Statistics , 1992 .

[28]  R. Adler An introduction to continuity, extrema, and related topics for general Gaussian processes , 1990 .

[29]  in Advances in Neural Information Processing , 1989 .

[30]  G. Golub Matrix computations , 1983 .

[31]  L. Breiman Classification and regression trees , 1983 .

[32]  W. Meredith,et al.  Statistics and Data Analysis , 1974 .