Uncertainty Analysis for Computationally Expensive Models with Multiple Outputs

Bayesian MCMC calibration and uncertainty analysis for computationally expensive models is implemented using the SOARS (Statistical and Optimization Analysis using Response Surfaces) methodology. SOARS uses a radial basis function interpolator as a surrogate, also known as an emulator or meta-model, for the logarithm of the posterior density. To prevent wasteful evaluations of the expensive model, the emulator is built only on a high posterior density region (HPDR), which is located by a global optimization algorithm. The set of points in the HPDR where the expensive model is evaluated is determined sequentially by the GRIMA algorithm described in detail in another paper but outlined here. Enhancements of the GRIMA algorithm were introduced to improve efficiency. A case study uses an eight-parameter SWAT2005 (Soil and Water Assessment Tool) model where daily stream flows and phosphorus concentrations are modeled for the Town Brook watershed which is part of the New York City water supply. A Supplemental Material file available online contains additional technical details and additional analysis of the Town Brook application.

[1]  Christine A. Shoemaker,et al.  Watershed calibration using multistart local optimization and evolutionary optimization with radial basis function approximation , 2007 .

[2]  Hans-Georg Frede,et al.  SWAT-G, a version of SWAT99.2 modified for application to low mountain range catchments , 2002 .

[3]  D. Ruppert,et al.  Transformation and Weighting in Regression , 1988 .

[4]  James O. Berger,et al.  A Framework for Validation of Computer Models , 2007, Technometrics.

[5]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[6]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[7]  Douglas M. Bates,et al.  Nonlinear Regression Analysis and Its Applications , 1988 .

[8]  David Ruppert,et al.  Power Transformations When Fitting Theoretical Models to Data , 1984 .

[9]  Katya Scheinberg,et al.  Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points , 2009, SIAM J. Optim..

[10]  H. Tjelmeland,et al.  Mode Jumping Proposals in MCMC , 2001 .

[11]  Peter Z. G. Qian,et al.  Bayesian Hierarchical Modeling for Integrating Low-Accuracy and High-Accuracy Experiments , 2008, Technometrics.

[12]  R. Clay Sprowls,et al.  Basic Statistical Concepts. , 1956 .

[13]  Fayçal Bouraoui,et al.  Modelling diffuse emission and retention of nutrients in the Vantaanjoki watershed (Finland) using the SWAT model , 2003 .

[14]  B. Ripley,et al.  Semiparametric Regression: Preface , 2003 .

[15]  Dave Higdon,et al.  Combining Field Data and Computer Simulations for Calibration and Prediction , 2005, SIAM J. Sci. Comput..

[16]  John R. Williams,et al.  LARGE AREA HYDROLOGIC MODELING AND ASSESSMENT PART I: MODEL DEVELOPMENT 1 , 1998 .

[17]  Christine A. Shoemaker,et al.  Local Derivative-Free Approximation of Computationally Expensive Posterior Densities , 2012, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[18]  A. P. Dawid,et al.  Gaussian Processes to Speed up Hybrid Monte Carlo for Expensive Bayesian Integrals , 2003 .

[19]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[20]  Bryan A. Tolson,et al.  Dynamically dimensioned search algorithm for computationally efficient watershed model calibration , 2007 .

[21]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[22]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[23]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .

[24]  Frank Vanden Berghen,et al.  CONDOR, a new parallel, constrained extension of Powell's UOBYQA algorithm: experimental results and comparison with the DFO algorithm , 2005 .

[25]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[26]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[27]  Stefan M. Wild,et al.  Bayesian Calibration and Uncertainty Analysis for Computationally Expensive Models Using Optimization and Radial Basis Function Approximation , 2008 .

[28]  G. Milliken.  Nonlinear Regression Analysis and Its Applications , 1990 .

[29]  Christine A. Shoemaker,et al.  Cannonsville Reservoir Watershed SWAT2000 model development, calibration and validation , 2007 .

[30]  Christine A. Shoemaker,et al.  Parallel Stochastic Global Optimization Using Radial Basis Functions , 2009, INFORMS J. Comput..

[31]  Christine A. Shoemaker,et al.  Watershed modeling of the Cannonsville Basin using SWAT2000: Model , 2004 .

[32]  Christine A. Shoemaker,et al.  Global Convergence of Radial Basis Function Trust Region Derivative-Free Algorithms , 2011, SIAM J. Optim..

[33]  M. J. Bayarri,et al.  Computer model validation with functional output , 2007, 0711.3271.

[34]  Thomas J. Santner,et al.  Sequential Design of Computer Experiments for Constrained Optimization , 2010 .

[35]  Christine A Shoemaker,et al.  Efficient Interpolation of Computationally Expensive Posterior Densities With Variable Parameter Costs , 2011, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[36]  Peter Z. G. Qian.  Nested Latin hypercube designs , 2009 .

[37]  D. Steinberg,et al.  Computer experiments: a review , 2010 .

[38]  Andrew D. Back,et al.  Radial Basis Functions , 2001 .

[39]  Michael Goldstein,et al.  Small Sample Bayesian Designs for Complex High-Dimensional Models Based on Information Gained Using Fast Approximations , 2009, Technometrics.