Emulation and Uncertainty Quantification for Models with Functional Response Using Bayesian Adaptive Splines

Author(s): Francom, Devin | Advisor(s): Sanso, Bruno | Abstract: When a computer code is used to simulate a complex system, a fundamental task is to assess the uncertainty of the simulator. In the case of computationally expensive simulators, this is often accomplished via a surrogate statistical model, a statistical output emulator. An effective emulator is one that provides good approximations to the computer code output for wide ranges of input values. In addition, an emulator should be able to handle large dimensional simulation output for a relevant number of inputs; it should flexibly capture heterogeneities in the variability of the response surface; it should be fast to evaluate for arbitrary combinations of input parameters; and it should provide an accurate quantification of the emulation uncertainty. In this work, we develop Bayesian adaptive spline methods for emulation of computer models that output functions. We introduce modifications to traditional Bayesian adaptive spline approaches that allow for fitting large amounts of data and allow for more efficient Markov chain Monte Carlo sampling. We develop a functional approach to sensitivity analysis that can be performed using this emulator. We present a sensitivity analysis of a computer model of the deformation of a protective plate used in pressure driven experiments. This example serves as an illustration of the ability of Bayesian adaptive spline emulators to fulfill all the necessities of computability, flexibility and reliable calculation on relevant measures of sensitivity.We extend the methods to emulation of an atmospheric dispersion simulator that outputs a plume in space and time based on inputs detailing the characteristics of the release, some of which are categorical. We achieve accurate emulation using Bayesian adaptive splines to model weights on empirical orthogonal functions. We extend the adaptive spline methodology to allow for categorical inputs. We use this emulator as well as appropriately identifiable simulator discrepancy and observational error models to calibrate the simulator using a dataset from an experimental release of particles from the Diablo Canyon Nuclear Power Plant in Central California. Since the release was controlled, these characteristics are known, allowing us to compare our findings to the truth.We further extend the methods to emulate a computer model that outputs misaligned functional data. We do this by modeling the aligned, or warped, data as well as the warping functions, using separate Bayesian adaptive spline models. We explore inference methods that treat these models jointly and separately, and establish methods to ensure that the warping functions are non-decreasing. These methods are applied to a high-energy-density physics model that outputs a curve representing energy as a function of time.

[1]  R. Gramacy,et al.  Categorical Inputs, Sensitivity Analysis, Optimization and Importance Tempering with tgp Version 2, an R Package for Treed Gaussian Process Models , 2010 .

[2]  Chao A. Hsiung,et al.  Shape restricted regression with random Bernstein polynomials , 2007, 0708.1054.

[3]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[4]  M. West,et al.  Shotgun Stochastic Search for “Large p” Regression , 2007 .

[5]  Paola Annoni,et al.  Sixth International Conference on Sensitivity Analysis of Model Output How to avoid a perfunctory sensitivity analysis , 2010 .

[6]  James O. Berger,et al.  A Framework for Validation of Computer Models , 2007, Technometrics.

[7]  Curtis B. Storlie,et al.  Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA , 2014, 1405.5297.

[8]  Jerome H. Friedman,et al.  Adaptive Spline Networks , 1990, NIPS.

[9]  Robert B. Gramacy,et al.  Cases for the nugget in modeling computer experiments , 2010, Statistics and Computing.

[10]  Derek Bingham,et al.  Efficient emulators of computer experiments using compactly supported correlation functions, with an application to cosmology , 2011, 1107.0749.

[11]  Christopher C. Holmes,et al.  Classification with Bayesian MARS , 2004, Machine Learning.

[12]  Hanoch Lev-Ari,et al.  Efficient solution of linear matrix equations with application to multistatic antenna array processing , 2005, Commun. Inf. Syst..

[13]  David J. Nott,et al.  Efficient sampling schemes for Bayesian MARS models with many predictors , 2005, Stat. Comput..

[14]  H. F. Stripling,et al.  Spline-Based Emulators for Radiative Shock Experiments With Measurement Error , 2013 .

[15]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[16]  Jeffrey S. Racine,et al.  Spline Regression in the Presence of Categorical Predictors , 2015 .

[17]  P. Lewis,et al.  Nonlinear Modeling of Time Series Using Multivariate Adaptive Regression Splines (MARS) , 1991 .

[18]  Donald D. Lucas,et al.  Bayesian inverse modeling of the atmospheric transport and emissions of a controlled tracer release from a nuclear power plant , 2017 .

[19]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[20]  Henry P. Wynn,et al.  Screening, predicting, and computer experiments , 1992 .

[21]  Kelli Humbird,et al.  Zonal flow generation in inertial confinement fusion implosions , 2017 .

[22]  David J. Pannell,et al.  Sensitivity Analysis of Normative Economic Models: Theoretical Framework and Practical Strategies , 1997 .

[23]  Sebastian Kurtek,et al.  Bayesian Model-Based Automatic Landmark Detection for Planar Curves , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[24]  Matieyendou Lamboni,et al.  Multivariate global sensitivity analysis for dynamic crop models , 2009 .

[25]  Shreyes N. Melkote,et al.  Analysis of Computer Experiments With Functional Response , 2012, Technometrics.

[26]  Ruichen Jin,et al.  Analytical Variance-Based Global Sensitivity Analysis in Simulation-Based Design Under Uncertainty , 2005, DAC 2004.

[27]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[28]  Sandhya Dwarkadas,et al.  Parallel Metropolis coupled Markov chain Monte Carlo for Bayesian phylogenetic inference , 2002, Bioinform..

[29]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[30]  A. Stohl,et al.  Technical note: The Lagrangian particle dispersion model FLEXPART version 6.2 , 2005 .

[31]  P. Green Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[32]  Radford M. Neal Sampling from multimodal distributions using tempered transitions , 1996, Stat. Comput..

[33]  Wei Wu,et al.  Generative models for functional data using phase and amplitude separation , 2012, Comput. Stat. Data Anal..

[34]  T. Kolda Multilinear operators for higher-order decompositions , 2006 .

[35]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[36]  James O. Berger,et al.  Modularization in Bayesian analysis, with emphasis on analysis of computer models , 2009 .

[37]  Fabio Rigat,et al.  Parallel hierarchical sampling: A general-purpose interacting Markov chains Monte Carlo algorithm , 2012, Comput. Stat. Data Anal..

[38]  D. Telesca Bayesian Analysis of Curves Shape Variation through Registration and Regression , 2015 .

[39]  岩崎 民子 SOURCES AND EFFECTS OF IONIZING RADIATION : United Nations Scientific Committee on the Effects of Atomic Radiation UNSCEAR 2000 Report to the General Assembly, with Scientific Annexes , 2002 .

[40]  Michael L. Littman,et al.  Bayesian Adaptive Sampling for Variable Selection and Model Averaging , 2011 .

[41]  G. Powers,et al.  A Description of the Advanced Research WRF Version 3 , 2008 .

[42]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models , 2004 .

[43]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[44]  Giles Hooker,et al.  Adapted Variational Bayes for Functional Data Registration, Smoothing, and Prediction , 2015, 1502.00552.

[45]  Bruno Sudret,et al.  Global sensitivity analysis using polynomial chaos expansions , 2008, Reliab. Eng. Syst. Saf..

[46]  Gareth O. Roberts,et al.  Towards optimal scaling of metropolis-coupled Markov chain Monte Carlo , 2011, Stat. Comput..

[47]  Jon C. Helton,et al.  Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models , 2009, Reliab. Eng. Syst. Saf..

[48]  I. Sobol Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[49]  Stefano Tarantola,et al.  Sensitivity Analysis as an Ingredient of Modeling , 2000 .

[50]  Marc G. Genton,et al.  Adjusted functional boxplots for spatio‐temporal data visualization and outlier detection , 2012 .

[51]  Richard H. Thuillier,et al.  Evaluation of a puff dispersion model in complex terrain , 1992 .

[52]  D. Higdon,et al.  Computer Model Calibration Using High-Dimensional Output , 2008 .

[53]  Robert B. Gramacy,et al.  Calibrating a large computer experiment simulating radiative shock hydrodynamics , 2014, 1410.3293.

[54]  Valerio Pascucci,et al.  ADAPTIVE SAMPLING WITH TOPOLOGICAL SCORES , 2013 .

[55]  M. J. Bayarri,et al.  Computer model validation with functional output , 2007, 0711.3271.

[56]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[57]  H. Chipman,et al.  Bayesian CART Model Search , 1998 .

[58]  Erica M. Rutter,et al.  A calibration and data assimilation method using the Bayesian MARS emulator , 2013 .