Estimation of Total Uncertainty in Modeling and Simulation

This report develops a general methodology for estimating the total uncertainty in computational simulations that deal with the numerical solution of a system of partial differential equations. A comprehensive, new view of the general phases of modeling and simulation is proposed, consisting of the following phases: conceptual modeling of the physical system, mathematical modeling of the conceptual model, discretization and algorithm selection for the mathematical model, computer programming of the discrete model, numerical solution of the computer program model, and representation of the numerical solution. Our view incorporates the modeling and simulation phases that are recognized in the operations research community, but it adds phases that are specific to the numerical solution of partial differential equations. In each of these phases, general sources of variability, uncertainty, and error are identified. Our general methodology is applicable to any discretization procedure for solving ordinary or partial differential equations. To demonstrate this methodology, we describe two system-level examples: a weapon involved in an aircraft crash-and-burn accident, and an unguided, rocket-boosted, aircraft-launched missile. The weapon in a crash and fire is discussed conceptually, but no computational simulations are performed. The missile flight example is discussed in more detail and computational results are presented.

[1]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[2]  Scott Ferson,et al.  What Monte Carlo methods cannot do , 1996 .

[3]  Madan M. Gupta,et al.  Analysis and management of uncertainty : theory and applications , 1992 .

[4]  Jon C. Helton,et al.  Guest editorial: treatment of aleatory and epistemic uncertainty in performance assessments for complex systems , 1996 .

[5]  Janina Decker,et al.  Simulation and Model-Based Methodologies: An Integrative View , 1984, NATO ASI Series.

[6]  N. Siu,et al.  Proceedings of workshop I in advanced topics in risk and reliability analysis. Model uncertainty: Its characterization and quantification , 1994 .

[7]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[8]  Les Hatton,et al.  The T-experiments: errors in scientific software , 1996, Quality of Numerical Software.

[9]  Hiromitsu Kumamoto,et al.  Probabilistic Risk Assessment and Management for Engineers and Scientists , 1996 .

[10]  Alan H. Karp,et al.  A variable order Runge-Kutta method for initial value problems with rapidly varying right-hand sides , 1990, TOMS.

[11]  P. Walley.  Statistical Reasoning with Imprecise Probabilities , 1990 .

[12]  Laurence Tianruo Yang,et al.  Fuzzy Logic with Engineering Applications , 1999 .

[13]  Earl Cox,et al.  The fuzzy systems handbook - a practitioner's guide to building, using, and maintaining fuzzy systems , 1994 .

[14]  I. A. Chaikovsky,et al.  Terminology for model credibility , 1979 .

[15]  George J. Klir,et al.  Fuzzy sets, uncertainty and information , 1988 .

[16]  M. V. Frank.  Treatment of uncertainties in space nuclear risk assessment with examples from Cassini mission applications , 1999 .

[17]  Miguel R. Visbal,et al.  On spurious behavior of CFD simulations , 1999 .

[18]  Roger U. Fujii.  Independent Verification and Validation , 2002 .

[19]  Ronald L. Iman,et al.  Expert opinion in risk analysis: the NUREG-1150 methodology , 1989 .

[20]  J. W. Garner,et al.  Uncertainty and sensitivity analysis results obtained in a preliminary performance assessment for the Waste Isolation Pilot Plant , 1993 .

[21]  D. Bell,et al.  Evidence Theory and Its Applications , 1991 .

[22]  M. B. Beck,et al.  Water quality modeling: A review of the analysis of uncertainty , 1987 .

[23]  F. O. Hoffman,et al.  An Examination of Response-Surface Methodologies for Uncertainty Analysis in Assessment Models , 1985 .

[24]  Peter L. Knepell,et al.  Simulation Validation: A Confidence Assessment Methodology , 1993 .

[25]  Hugh W. Coleman,et al.  Experimentation and Uncertainty Analysis for Engineers , 1989 .

[26]  Ray J. Paul,et al.  Computer Simulation and Modelling , 1987 .

[27]  Claes Johnson.  Numerical solution of partial differential equations by the finite element method , 1988 .

[28]  M. Sebrechts.  Ignorance and Uncertainty: Emerging Paradigms , 1989 .

[29]  D. P. Gallegos,et al.  Components of an overall performance assessment methodology , 1990 .

[30]  Ove Ditlevsen,et al.  Uncertainty modeling with applications to multidimensional civil engineering systems , 1981 .

[31]  I. Catton,et al.  Quantifying reactor safety margins part 1: An overview of the code scaling, applicability, and uncertainty evaluation methodology , 1990 .

[32]  Kenneth F. Alvin,et al.  A New Methodology for the Estimation of Total Uncertainty in Computational Simulation , 1999 .

[33]  Madan M. Gupta,et al.  Uncertainty modelling and analysis : theory and applications , 1994 .

[34]  Shesh N. Rai,et al.  A general framework for the analysis of uncertainty and variability in risk assessment , 1996 .

[35]  T. Vamos.  Epistemic background problems of uncertainty , 1990, [1990] Proceedings. First International Symposium on Uncertainty Modeling and Analysis.

[36]  Osman Balci.  Guidelines for successful simulation studies , 1990, 1990 Winter Simulation Conference Proceedings.

[37]  Earl Cox,et al.  The fuzzy systems handbook , 1994 .

[38]  G. J. Klir,et al.  Probabilistic versus possibilistic conceptualization of uncertainty , 1990, [1990] Proceedings. First International Symposium on Uncertainty Modeling and Analysis.

[39]  Jon C. Helton,et al.  Summary description of the methods used in the probabilistic risk assessments for NUREG-1150 , 1992 .

[40]  Engineering Risks: Evaluation and Valuation , 1991 .

[41]  Russell G. Almond.  Graphical belief modeling , 1995 .

[42]  H. Frey,et al.  Characterizing, simulating, and analyzing variability and uncertainty: An illustration of methods using an air toxics emissions example , 1996 .

[43]  R. Deal.  Simulation Modeling and Analysis (2nd Ed.) , 1994 .

[44]  Edward Tenner,et al.  Why Things Bite Back , 1997 .

[45]  Rachelle D. Hollander,et al.  Acceptable Evidence: Science and Values in Risk Management , 1994 .

[46]  R. Spear,et al.  Integrating uncertainty and interindividual variability in environmental risk assessment. , 1987, Risk analysis : an official publication of the Society for Risk Analysis.

[47]  Gareth W. Parry,et al.  The characterization of uncertainty in probabilistic risk assessments of complex systems , 1996 .

[48]  Richard E. Nance.  Model Representation in Discrete Event Simulation: The Conical Methodology , 1981 .

[49]  H. C. Yee,et al.  Aspects of Numerical Uncertainties in Time Marching to Steady-State Numerical Solutions , 1998 .

[50]  Janusz S. Kowalik,et al.  Mathematical modeling with computers , 1980 .

[51]  R. Iman,et al.  A distribution-free approach to inducing rank correlation among input variables , 1982 .

[52]  C. Julius Wang,et al.  What Every Engineer Should Know about Reliability and Risk Analysis , 1993 .

[53]  Bilal M. Ayyub,et al.  Uncertainty Modeling and Analysis in Civil Engineering , 1997 .

[54]  L. R. Rollstin,et al.  Aeroballistic and mechanical design and development of the Talos-Terrier-Recruit (TATER) rocket system with flight test results , 1976 .

[55]  William L. Oberkampf,et al.  Issues in Computational Fluid Dynamics Code Verification and Validation , 1997 .

[56]  William H. Press,et al.  Numerical Recipes: FORTRAN , 1988 .

[57]  P. Kloeden,et al.  Numerical Solution of Stochastic Differential Equations , 1992 .

[58]  George E. P. Box,et al.  Sampling and Bayes' inference in scientific modelling and robustness , 1980 .

[59]  Miguel R. Visbal,et al.  On spurious behavior of CFD simulations , 1997 .

[60]  William L. Oberkampf,et al.  Uncertainty and error in computational simulations , 1997 .

[61]  P. R. Ballin,et al.  Quantifying the Impact of Geological Uncertainty on Reservoir Performing Forecasts , 1993 .

[62]  Arie Tzvieli.  Possibility theory: An approach to computerized processing of uncertainty , 1990, J. Am. Soc. Inf. Sci..

[63]  Tony L. Wahl,et al.  Experimentation and Uncertainty Analysis for Engineers, 2nd Ed. , 1999 .

[64]  G. Klir,et al.  Uncertainty-based information: Elements of generalized information theory (studies in fuzziness and soft computing). , 1998 .

[65]  Dominic A. Clark,et al.  Representing uncertain knowledge - an artificial intelligence approach , 1993 .

[66]  M. Elisabeth Paté-Cornell,et al.  Uncertainties in risk analysis: Six levels of treatment , 1996 .

[67]  Alfredo Ang H.-S.,et al.  Probability concepts in engineering planning and design, vol i : basic principles , 1979 .

[68]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[69]  Robert L. Mullen,et al.  Development of interval based methods for fuzziness in continuum-mechanics , 1995, Proceedings of 3rd International Symposium on Uncertainty Modeling and Analysis and Annual Conference of the North American Fuzzy Information Processing Society.

[70]  James H. Lambert,et al.  When and How Can You Specify a Probability Distribution When You Don't Know Much? II , 1999 .

[71]  S. Ferson,et al.  Different methods are needed to propagate ignorance and variability , 1996 .

[72]  J C Helton,et al.  Performance Assessment in Support of the 1996 Compliance Certification Application for the Waste Isolation Pilot Plant , 1999, Risk analysis : an official publication of the Society for Risk Analysis.

[73]  P. A. Davis,et al.  Uncertainties associated with performance assessment of high-level radioactive waste repositories , 1990 .

[74]  Robert G. Sargent,et al.  An expository on verification and validation of simulation models , 1985, WSC '85.

[75]  H. C. Yee,et al.  DYNAMICAL APPROACH STUDY OF SPURIOUS STEADY-STATE NUMERICAL SOLUTIONS OF NONLINEAR DIFFERENTIAL EQUATIONS II. GLOBAL ASYMPTOTIC BEHAVIOR OF TIME DISCRETIZATIONS ∗ , 1995 .

[76]  Hartmut Bossel,et al.  Modeling and simulation , 1994 .

[77]  H. Christopher Frey,et al.  Probabilistic Techniques in Exposure Assessment: A Handbook for Dealing with Variability and Uncertainty in Models and Inputs , 1999 .

[78]  L. Richardson.  The Approximate Arithmetical Solution by Finite Differences of Physical Problems Involving Differential Equations, with an Application to the Stresses in a Masonry Dam , 1911 .

[79]  Wilson H. Tang,et al.  Probability concepts in engineering planning and design , 1984 .

[80]  Da Ruan,et al.  Foundations and Applications of Possibility Theory , 1995 .

[81]  Stephen C. Hora,et al.  Aleatory and epistemic uncertainty in probability elicitation with an example from hazardous waste management , 1996 .

[82]  Derek J. Pike,et al.  Empirical Model‐building and Response Surfaces. , 1988 .

[83]  Igor Kozine,et al.  Imprecise Probabilities Relating to Prior Reliability Assessments , 1999, ISIPTA.

[84]  F. O. Hoffman,et al.  Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. , 1994, Risk analysis : an official publication of the Society for Risk Analysis.

[85]  H. C. Yee,et al.  GLOBAL ASYMPTOTIC BEHAVIOR OF ITERATIVE IMPLICIT SCHEMES , 1994 .

[86]  Jon C. Helton,et al.  Treatment of Uncertainty in Performance Assessments for Complex Systems , 1994 .