Gaussian Process Single-Index Models as Emulators for Computer Experiments

A single-index model (SIM) provides for parsimonious multidimensional nonlinear regression by combining parametric (linear) projection with univariate nonparametric (nonlinear) regression models. We show that a particular Gaussian process (GP) formulation is simple to work with and ideal as an emulator for some types of computer experiment as it can outperform the canonical separable GP regression model commonly used in this setting. Our contribution focuses on drastically simplifying, reinterpreting, and then generalizing a recently proposed fully Bayesian GP-SIM combination. Favorable performance is illustrated on synthetic data and a real-data computer experiment. Two R packages, both released on CRAN, have been augmented to facilitate inference under our proposed model(s).

[1]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo in Practice: A Roundtable Discussion , 1998 .

[2]  Robert B. Gramacy,et al.  Particle Learning of Gaussian Process Models for Sequential Design and Optimization , 2009, 0909.5262.

[3]  Stanley H. Cohen,et al.  Design and Analysis , 2010 .

[4]  Edward I. George,et al.  Bayesian Treed Models , 2002, Machine Learning.

[5]  Anestis Antoniadis,et al.  BAYESIAN ESTIMATION IN SINGLE-INDEX MODELS , 2004 .

[6]  Dave Higdon,et al.  Combining Field Data and Computer Simulations for Calibration and Prediction , 2005, SIAM J. Sci. Comput..

[7]  Robert B. Gramacy,et al.  Optimization Under Unknown Constraints , 2010, 1004.4027.

[8]  Robert B. Gramacy,et al.  Bayesian treed gaussian process models , 2005 .

[9]  D. Brillinger.  A Generalized Linear Model With “Gaussian” Regressor Variables , 2012 .

[10]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[11]  Robert B. Gramacy,et al.  Ja n 20 08 Bayesian Treed Gaussian Process Models with an Application to Computer Modeling , 2009 .

[12]  T. Choi,et al.  A Gaussian process regression approach to a single-index model , 2011 .

[13]  D. Brillinger.  The identification of a particular nonlinear time series system , 1977 .

[14]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[15]  Robert B. Gramacy,et al.  Cases for the nugget in modeling computer experiments , 2010, Statistics and Computing.

[16]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[17]  Y. Xia,et al.  A Multiple-Index Model and Dimension Reduction , 2008 .

[18]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[19]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[20]  A. P. Dawid,et al.  Regression and Classification Using Gaussian Process Priors , 2009 .

[21]  T. J. Mitchell,et al.  Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction , 1993 .

[22]  Anthony O'Hagan,et al.  Diagnostics for Gaussian Process Emulators , 2009, Technometrics.

[23]  B. A. Worley.  Deterministic uncertainty analysis , 1987 .

[24]  Robert B. Gramacy,et al.  tgp: An R Package for Bayesian Nonstationary, Semiparametric Nonlinear Regression and Design by Treed Gaussian Process Models , 2007 .

[25]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[26]  Peter S. Craig,et al.  A new reconstruction of multivariate normal orthant probabilities , 2007 .

[27]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[28]  Nicholas G. Polson,et al.  Particle Learning and Smoothing , 2010, 1011.1098.

[29]  Robert B. Gramacy,et al.  Classification and Categorical Inputs with Treed Gaussian Process Models , 2009, J. Classif..

[30]  R. Gramacy,et al.  Categorical Inputs, Sensitivity Analysis, Optimization and Importance Tempering with tgp Version 2, an R Package for Treed Gaussian Process Models , 2010 .

[31]  Robert B. Gramacy,et al.  Adaptive Design and Analysis of Supercomputer Experiments , 2008, Technometrics.

[32]  Hai-Bin Wang,et al.  Bayesian estimation and variable selection for single index models , 2009, Comput. Stat. Data Anal..

[33]  H. Ichimura,et al.  SEMIPARAMETRIC LEAST SQUARES (SLS) AND WEIGHTED SLS ESTIMATION OF SINGLE-INDEX MODELS , 1993 .

[34]  Anthony J. Hayter,et al.  The evaluation of general non‐centred orthant probabilities , 2003 .

[35]  Klaus Obermayer,et al.  Gaussian process regression: active data selection and test point rejection , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.