Calibration and Uncertainty Analysis of Predictions from Computational Models

Abstract: Computer experiments are simulations of physical experiments performed by exercising a mathematical model for a physical or chemical process, to produce model outputs corresponding to sets of values of inputs to the model. They are especially useful when the corresponding physical experiments are difficult or expensive. In these cases one will perform a relatively small number of physical experiments that are used to calibrate the simulation model. This is then employed to explore the space of values of the inputs extensively, in a way that “interpolates” the results of the physical experiments. Quantification of uncertainty for the outputs of such computer experiments is of great interest. The sources of uncertainty are in part due to the experimental measurement uncertainty in the inputs, and in part due to inadequacies of the underlying mathematical model. This paper presents methods for calibration and assessment of both types of uncertainty and demonstrates their use on two simple fire models.

[1]  T. J. Mitchell,et al.  Bayesian Prediction of Deterministic Functions, with Applications to the Design and Analysis of Computer Experiments , 1991 .

[2]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[3]  R. Cordero,et al.  Evaluating the uncertainties of data rendered by computational models , 2007 .

[4]  Andrew L. Rukhin,et al.  Three Statistical Paradigms for the Assessment and Interpretation of Measurement Uncertainty , 2009 .

[5]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[6]  Blaza Toman,et al.  Assessment of measurement uncertainty via observation equations , 2007 .

[7]  Robert G. Easterling,et al.  Measuring the Predictive Capability of Computational Models: Principles and Methods, Issues and Illustrations , 2001 .

[8]  Michael Goldstein,et al.  Bayesian Forecasting for Complex Systems Using Computer Simulators , 2001 .

[9]  B. J. McCaffrey,et al.  Estimating room temperatures and the likelihood of flashover using fire test data correlations , 1981 .

[10]  James O. Berger,et al.  A Framework for Validation of Computer Models , 2007, Technometrics.

[11]  C. L. Beyler,et al.  Analysis Of Compartment Fires With Overhead Forced Ventilation , 1991 .

[12]  Michael Goldstein,et al.  Probabilistic Formulations for Transferring Inferences from Mathematical Models to Physical Systems , 2005, SIAM J. Sci. Comput..

[13]  E. Iso,et al.  Measurement Uncertainty and Probability: Guide to the Expression of Uncertainty in Measurement , 1995 .

[14]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .