Latin Hypercube Sampling and the Propagation of Uncertainty in Analyses of Complex Systems

The following techniques for uncertainty and sensitivity analysis are briefly summarized: Monte Carlo analysis, differential analysis, response surface methodology, Fourier amplitude sensitivity test, Sobol’ variance decomposition, and fast probability integration. Desirable features of Monte Carlo analysis in conjunction with Latin hypercube sampling are described in discussions of the following topics: (i) properties of random, stratified and Latin hypercube sampling, (ii) comparisons of random and Latin hypercube sampling, (iii) operations involving Latin hypercube sampling (i.e. correlation control, reweighting of samples to incorporate changed distributions, replicated sampling to test reproducibility of results), (iv) uncertainty analysis (i.e. cumulative distribution functions, complementary cumulative distribution functions, box plots), (v) sensitivity analysis (i.e. scatterplots, regression analysis, correlation analysis, rank transformations, searches for nonrandom patterns), and (vi) analyses involving stochastic (i.e. aleatory) and subjective (i.e. epistemic) uncertainty. Published by Elsevier Science Ltd.

[1]  L. Goossens,et al.  Expert judgement for a probabilistic accident consequence uncertainty analysis , 2000 .

[2]  C. Fortuin,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory , 1973 .

[3]  James R. Wilson,et al.  Integrated Variance Reduction Strategies for Simulation , 1996, Oper. Res..

[4]  M. Stein.  Large sample properties of simulations using latin hypercube sampling , 1987 .

[5]  A. C. Payne,et al.  Analysis of the LaSalle Unit 2 Nuclear Power Plant: Risk Methods Integration and Evaluation Program (RMIEP). Volume 3, Part 2, Internal events accident sequence quantification: Appendices , 1992 .

[6]  Terry Andres.  Sampling methods and sensitivity analysis for large parameter sets , 1997 .

[7]  A. Owen.  A Central Limit Theorem for Latin Hypercube Sampling , 1992 .

[8]  Nancy J. Brown,et al.  First- and Second-Order Sensitivity Analysis of a Photochemically Reactive System (a Green's Function Approach) , 1997 .

[9]  J. D. Morrison,et al.  Evaluating prediction uncertainty in simulation models , 1999 .

[10]  John W Tukey.  THE PROPAGATION OF ERRORS, FLUCTUATIONS AND TOLERANCES, SUPPLEMENTARY FORMULAS , 1957 .

[11]  Jeong‐Soo Park.  Optimal Latin-hypercube designs for computer experiments , 1994 .

[12]  Richard Kielbasa,et al.  Latin Hypercube Sampling Monte Carlo Estimation of Average Quality Index for Integrated Circuits , 1997 .

[13]  D. Pontier,et al.  A discrete-event computer model of feline herpes virus within cat populations. , 2000, Preventive Veterinary Medicine.

[14]  A. Saltelli,et al.  A quantitative model-independent method for global sensitivity analysis of model output , 1999 .

[15]  David C. Cox,et al.  An Analytic Method for Uncertainty Analysis of Nonlinear Output Functions, with Applications to Fault-Tree Analysis , 1982, IEEE Transactions on Reliability.

[16]  R. Barton,et al.  Factorial hypercube designs for spatial correlation regression , 1997 .

[17]  Peter J. Diggle,et al.  Simple Monte Carlo Tests for Spatial Pattern , 1977 .

[18]  Nathan Siu,et al.  Bayesian parameter estimation in probabilistic risk assessment , 1998 .

[19]  R. L. Iman,et al.  Rank correlation plots for use with correlated input variables in simulation studies , 1980 .

[20]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[21]  F. E. Haskin,et al.  Efficient uncertainty analyses using fast probability integration , 1996 .

[22]  Stefano Tarantola,et al.  Winding Stairs: A sampling tool to compute sensitivity indices , 2000, Stat. Comput..

[23]  Ali Mosleh,et al.  A critique of current practice for the use of expert opinions in probabilistic risk assessment , 1988 .

[24]  A. Saltelli,et al.  Non-parametric statistics in sensitivity analysis for model output: A comparison of selected techniques , 1990 .

[25]  D. Teichroew.  A History of Distribution Sampling Prior to the Era of the Computer and its Relevance to Simulation , 1965 .

[26]  Byung Hwan Oh,et al.  Sensitivity Analysis of Time-Dependent Behavior in PSC Box Girder Bridges , 2000 .

[27]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[28]  Jon C. Helton,et al.  Direct releases to the surface and associated complementary cumulative distribution functions in the 1996 performance assessment for the Waste Isolation Pilot Plant: cuttings, cavings and spallings , 2000, Reliab. Eng. Syst. Saf..

[29]  Jon C. Helton,et al.  Characterization of subjective uncertainty in the 1996 performance assessment for the Waste Isolation Pilot Plant , 2000, Reliab. Eng. Syst. Saf..

[30]  Ronald L. Iman,et al.  The Repeatability of Uncertainty and Sensitivity Analyses for Complex Probabilistic Risk Assessments , 1991 .

[31]  Katherine Campbell,et al.  Setting Environmental Standards: The Statistical Approach to Handling Uncertainty and Variation , 1997 .

[32]  Peter Mathé,et al.  Hilbert space analysis of Latin Hypercube Sampling , 2000 .

[33]  Boxin Tang.  A theorem for selecting OA-based Latin hypercubes using a distance criterion , 1994 .

[34]  B. Ripley.  Tests of 'Randomness' for Spatial Point Patterns , 1979 .

[35]  G. Prabhakar Rao,et al.  Sensitivity studies of air scattered neutron dose from particle accelerators , 1997 .

[36]  K. Shuler,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. II Applications , 1973 .

[37]  Jon C. Helton,et al.  Sampling-based methods for uncertainty and sensitivity analysis. , 2000 .

[38]  H. Rabitz,et al.  General foundations of high‐dimensional model representations , 1999 .

[39]  Eva Riccomagno,et al.  Experimental Design and Observation for Large Systems , 1996, Journal of the Royal Statistical Society: Series B (Methodological).

[40]  McKay,et al.  Report on the application of statistical techniques to the analysis of computer codes , 1976 .

[41]  Paul I. Barton,et al.  On computational differentiation , 1998 .

[42]  Martin Berz,et al.  Computational differentiation : techniques, applications, and tools , 1996 .

[43]  J. Sacks,et al.  Design and analysis of numerical experiments , 1993 .

[44]  G. Heuvelink,et al.  Latin Hypercube Sampling of Gaussian Random Fields , 1999, Technometrics.

[45]  R. Rackwitz,et al.  Structural reliability under combined random load sequences , 1978 .

[46]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[47]  Stan Kaplan,et al.  ‘Expert information’ versus ‘expert opinions’. Another approach to the problem of eliciting/ combining/using expert knowledge in PRA , 1992 .

[48]  C. T. Haan.  Parametric Uncertainty in Hydrologic Modeling , 1989 .

[49]  R. Cooke.  Experts in Uncertainty: Opinion and Subjective Probability in Science , 1991 .

[50]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[51]  C. M. Stone,et al.  SANTOS - a two-dimensional finite element program for the quasistatic, large deformation, inelastic response of solids , 1997 .

[52]  Akimichi Takemura,et al.  On reduction of finite-sample variance by extended Latin hypercube sampling , 2000 .

[53]  S J Finch,et al.  Comparison of empirical strategies to maximize GENEHUNTER lod scores , 1999, Genetic epidemiology.

[54]  Kenny Q. Ye,et al.  Algorithmic construction of optimal symmetric Latin hypercube designs , 2000 .

[55]  I. Sobol,et al.  Sensitivity Measures, ANOVA-like Techniques and the Use of Bootstrap , 1997 .

[56]  Boxin Tang.  Orthogonal Array-Based Latin Hypercubes , 1993 .

[57]  John H. Seinfeld,et al.  Global sensitivity analysis—a computational implementation of the Fourier Amplitude Sensitivity Test (FAST) , 1982 .

[58]  Ezequiel V. Albano.  The Monte Carlo Simulation Method: a Powerful Tool for the Study of Reaction Processes , 1996 .

[59]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[60]  J. C. Helton,et al.  A COMPARISON OF UNCERTAINTY AND SENSITIVITY ANALYSIS TECHNIQUES FOR COMPUTER MODELS , 1985 .

[61]  W. B. Murfin,et al.  The NUREG-1150 probabilistic risk assessment for the Surry Nuclear Power Station☆ , 1992 .

[62]  Nikolay Ivanov Kolev,et al.  Uncertainty and sensitivity analysis of a postexperiment simulation of nonexplosive melt-water interaction , 1996 .

[63]  D. M. Hamby,et al.  Parameter uncertainty and sensitivity in a liquid-effluent dose model , 1995, Environmental monitoring and assessment.

[64]  Wei-Liem Loh.  On Latin hypercube sampling , 1996 .

[65]  A. M. Hasofer,et al.  Exact and Invariant Second-Moment Code Format , 1974 .

[66]  E. Gosset.  A three-dimensional extended Kolmogorov-Smirnov test as a useful tool in astronomy , 1987 .

[67]  Ralph L. Keeney,et al.  Eliciting probabilities from experts in complex technical problems , 1991 .

[68]  Y. Rubin,et al.  A methodology to integrate site characterization information into groundwater‐driven health risk assessment , 1999 .

[69]  Jon C. Helton,et al.  Performance Assessment for the Waste Isolation Pilot Plant: From Regulation to Calculation for 40 CFR 191.13 , 1997, Oper. Res..

[70]  Jon C. Helton,et al.  Risk, Uncertainty in Risk, and the EPA Release Limits for Radioactive Waste Disposal , 1993 .

[71]  Mike Burton,et al.  The effect of uncertainties in kinetic and photochemical data on model predictions of stratospheric ozone depletion , 1997 .

[72]  Ronald L. Wasserstein,et al.  Monte Carlo: Concepts, Algorithms, and Applications , 1997 .

[73]  J. Halton.  A Retrospective and Prospective Survey of the Monte Carlo Method , 1970 .

[74]  Jon C. Helton,et al.  Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal , 1993 .

[75]  Helge Hove Haldorsen,et al.  Putting error bars on reservoir engineering forecasts , 1992 .

[76]  J. Garvey,et al.  FROM STAR CHARTS TO STONEFLIES: DETECTING RELATIONSHIPS IN CONTINUOUS BIVARIATE DATA , 1998 .

[77]  K. Byth.  On Robust Distance-Based Intensity Estimators , 1982 .

[78]  Herschel Rabitz,et al.  The Green’s function method of sensitivity analysis in chemical kinetics , 1978 .

[79]  André I. Khuri,et al.  Response surface methodology: 1966–1988 , 1989 .

[80]  L H Goossens,et al.  Joint EC/USNRC expert judgement driven radiological protection uncertainty analysis. , 1998, Journal of radiological protection : official journal of the Society for Radiological Protection.

[81]  Doyle R. Edwards,et al.  Sensitivity Analysis of Thermal-Hydraulic Parameters and Probability Estimation of Boiling Transition in a Standard BWR/6 , 1991 .

[82]  P. Diggle,et al.  Some Distance-Based Tests of Independence for Sparsely-Sampled Multivariate Spatial Point Patterns , 1983 .

[83]  Jon C. Helton,et al.  Guest editorial: treatment of aleatory and epistemic uncertainty in performance assessments for complex systems , 1996 .

[84]  Ronald L. Iman,et al.  Risk methodology for geologic disposal of radioactive waste: small sample sensitivity analysis techniques for computer models, with an application to risk assessment , 1980 .

[85]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[86]  S. Kaplan,et al.  On The Quantitative Definition of Risk , 1981 .

[87]  Boxin Tang,et al.  SELECTING LATIN HYPERCUBES USING CORRELATION CRITERIA , 1998 .

[88]  Jon C. Helton,et al.  Robustness of an uncertainty and sensitivity analysis of early exposure results with the MACCS reactor accident consequence model , 1995 .

[89]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo Method , 1981 .

[90]  R. Assunção.  TESTING SPATIAL RANDOMNESS BY MEANS OF ANGLES , 1994 .

[91]  Rob P. Rechard,et al.  Historical background on performance assessment for the Waste Isolation Pilot Plant , 1999, Reliab. Eng. Syst. Saf..

[92]  J C Helton,et al.  Performance Assessment in Support of the 1996 Compliance Certification Application for the Waste Isolation Pilot Plant , 1999, Risk analysis : an official publication of the Society for Risk Analysis.

[93]  Ralph L. Keeney,et al.  Elicitation and use of expert judgment in performance assessment for high-level radioactive waste repositories , 1990 .

[94]  E Ackerman,et al.  Parameter sensitivity of a model of viral epidemics simulated with Monte Carlo techniques. II. Durations and peaks. , 1993, International journal of bio-medical computing.

[95]  C. M. Stone.  Final disposal room structural response calculations , 1997 .

[96]  Laura Toran,et al.  Subsurface stormflow modeling with sensitivity analysis using a Latin-hypercube sampling technique , 1996 .

[97]  Jack P. C. Kleijnen,et al.  Sensitivity analysis and related analyses: A review of some statistical techniques , 1997 .

[98]  R. H. Myers,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[99]  F. W. Whicker,et al.  Contaminant Transport through Agroecosystems: Assessing Relative Importance of Environmental, Physiological, and Management Factors. , 1992, Ecological applications : a publication of the Ecological Society of America.

[100]  Stephen D. Unwin,et al.  Controlling principles for prior probability assignments in nuclear risk assessment , 1986 .

[101]  Kenny Q. Ye.  Orthogonal Column Latin Hypercubes and Their Application in Computer Experiments , 1998 .

[102]  Robert W. Schanz,et al.  Evaluation of the Rackwitz-Fiessler Uncertainty Analysis Method for Environmental Fate and Transport Models , 1992 .

[103]  Gene H. Golub,et al.  Matrix computations , 1983 .

[104]  A. Saltelli,et al.  An alternative way to compute Fourier amplitude sensitivity test (FAST) , 1998 .

[105]  Jon C. Helton,et al.  Calculation of reactor accident safety goals , 1993 .

[106]  T. McKone.  Uncertainty and variability in human exposures to soil contaminants through home-grown food: a Monte Carlo assessment. , 1994, Risk analysis : an official publication of the Society for Risk Analysis.

[107]  M. D. McKay,et al.  Monte Carlo estimation under different distributions using the same simulation , 1987 .

[108]  H. Steinberg.  GENERALIZED QUOTA SAMPLING , 1963 .

[109]  Robert L. Winkler,et al.  Uncertainty in probabilistic risk assessment , 1996 .

[110]  Jon C. Helton,et al.  Summary discussion of the 1996 performance assessment for the Waste Isolation Pilot Plant , 2000, Reliab. Eng. Syst. Saf..

[111]  Melvin G. Marietta,et al.  Regulatory basis for the Waste Isolation Pilot Plant performance assessment , 2000, Reliab. Eng. Syst. Saf..

[112]  Jon C. Helton,et al.  Radionuclide and colloid transport in the Culebra Dolomite and associated complementary cumulative distribution functions in the 1996 performance assessment for the Waste Isolation Pilot Plant , 2000, Reliab. Eng. Syst. Saf..

[113]  H. Christopher Frey,et al.  Probabilistic Techniques in Exposure Assessment: A Handbook for Dealing with Variability and Uncertainty in Models and Inputs , 1999 .

[114]  T. J. Mitchell,et al.  Bayesian Prediction of Deterministic Functions, with Applications to the Design and Analysis of Computer Experiments , 1991 .

[115]  M. Mckay,et al.  Critique of and Limitations on the Use of Expert Judgements in Accident Consequence Uncertainty Analysis , 2000 .

[116]  G. Fasano,et al.  A multidimensional version of the Kolmogorov–Smirnov test , 1987 .

[117]  Jon C. Helton,et al.  Uncertainty and Sensitivity Analysis Results Obtained in the 1996 Performance Assessment for the Waste Isolation Pilot Plant , 1998 .

[118]  J. Kleijnen,et al.  Statistical analyses of scatterplots to identify important factors in large-scale simulations, 2: robustness of techniques , 1999 .

[119]  F. O. Hoffman,et al.  Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. , 1994, Risk analysis : an official publication of the Society for Risk Analysis.

[120]  M. Becker,et al.  Sensitivity and uncertainty analysis of reactor performance parameters , 1982 .

[121]  Jon C. Helton,et al.  Direct releases to the surface and associated complementary cumulative distribution functions in the 1996 performance assessment for the Waste Isolation Pilot Plant: direct brine release , 2000, Reliab. Eng. Syst. Saf..

[122]  M. C. Thorne,et al.  The use of expert opinion in formulating conceptual models of underground disposal systems and the treatment of associated bias , 1993 .

[123]  Jon C. Helton,et al.  Treatment of Uncertainty in Performance Assessments for Complex Systems , 1994 .

[124]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[125]  George Apostolakis,et al.  A taxonomy of issues related to the use of expert judgments in probabilistic safety studies , 1992 .

[126]  Jack P. C. Kleijnen,et al.  Techniques for sensitivity analysis of simulation models: A case study of the CO2 greenhouse effect , 1992, Simul..

[127]  Jon C. Helton,et al.  Evaluation of severe accident risks, Peach Bottom, Unit 2: Main report , 1990 .

[128]  T. D. Brown,et al.  The NUREG-1150 probabilistic risk assessment for the Grand Gulf Nuclear Station , 1992 .

[129]  D. Cacuci.  Sensitivity theory for nonlinear systems. II. Extensions to additional classes of responses , 1981 .

[130]  Michael D. McKay,et al.  Nonparametric variance-based methods of assessing uncertainty importance , 1997 .

[131]  Hal Caswell,et al.  HARBOR PORPOISE AND FISHERIES: AN UNCERTAINTY ANALYSIS OF INCIDENTAL MORTALITY , 1998 .

[132]  T. A. Wheeler,et al.  Use of expert judgment in NUREG--1150 , 1991 .

[133]  Eugene P. Dougherty,et al.  Further developments and applications of the Green’s function method of sensitivity analysis in chemical kinetics , 1979 .

[134]  William J. Hill,et al.  A Review of Response Surface Methodology: A Literature Survey* , 1966 .

[135]  M. Morris.  Three Technometrics experimental design classics , 2000 .

[136]  Yih‐Tsuen Wu,et al.  New Algorithm for Structural Reliability Estimation , 1987 .

[137]  W. V. Loscutoff,et al.  General sensitivity theory , 1972 .

[138]  R. Maxwell,et al.  Stochastic environmental risk analysis: an integrated methodology for predicting cancer risk from contaminated groundwater , 1999 .

[139]  J. C. Helton,et al.  Uncertainty and sensitivity analysis in performance assessment for the Waste Isolation Pilot Plant , 1999 .

[140]  James R. Wilson,et al.  Correlation-Induction Techniques for Estimating Quantiles in Simulation Experiments , 1998, Oper. Res..

[141]  C. Donnelly,et al.  Transmission dynamics and epidemiology of BSE in British cattle , 1996, Nature.

[142]  Raymond H. Myers,et al.  Response Surface Methodology--Current Status and Future Directions , 1999 .

[143]  Jack P. C. Kleijnen,et al.  Sensitivity analysis of simulation experiments: regression analysis and statistical design , 1992 .

[144]  M. Deaton,et al.  Response Surfaces: Designs and Analyses , 1989 .

[145]  H. Rief,et al.  Review of Monte Carlo Techniques for Analyzing Reactor Perturbations , 1986 .

[146]  James E. Campbell,et al.  An Approach to Sensitivity Analysis of Computer Models: Part I—Introduction, Input Variable Selection and Preliminary Variable Assessment , 1981 .

[147]  Wei-Liem Loh,et al.  A combinatorial central limit theorem for randomized orthogonal array sampling designs , 1996 .

[148]  R. Iman,et al.  A measure of top-down correlation , 1987 .

[149]  M. Kramer,et al.  Sensitivity Analysis in Chemical Kinetics , 1983 .

[150]  Karla Hoffman,et al.  Obtaining minimum-correlation Latin Hypercube Sampling plans using an IP-based heuristic , 1995 .

[151]  Terry Andres,et al.  Sensitivity analysis of model output: an investigation of new techniques , 1993 .

[152]  S. Blower,et al.  Uncertainty and sensitivity analysis of the basic reproductive rate. Tuberculosis as an example. , 1997, American journal of epidemiology.

[153]  T. Cox,et al.  A conditioned distance ratio method for analyzing spatial patterns , 1976 .

[154]  Hadi Dowlatabadi,et al.  Sensitivity and Uncertainty Analysis of Complex Models of Disease Transmission: an HIV Model, as an Example , 1994 .

[155]  Jane M. Booker,et al.  Eliciting and analyzing expert judgement - a practical guide , 2001, ASA-SIAM series on statistics and applied probability.

[156]  G. E. Apostolakis,et al.  Theoretical foundations and practical issues for using expert judgements in uncertainty analysis of high-level radioactive waste disposal , 1991 .

[157]  Christian Bischof,et al.  ADIC: an extensible automatic differentiation tool for ANSI-C , 1997 .

[158]  R. S. May,et al.  RESPONSE SURFACE DEVELOPMENT USING RETRAN , 1991 .

[159]  Kristen Lohman,et al.  A Probabilistic Analysis of Regional Mercury Impacts on Wildlife , 2000 .

[160]  Soon Heung Chang,et al.  A Procedure for Statistical Thermal Margin Analysis Using Response Surface Method and Monte Carlo Technique , 1986 .

[161]  J. Hammersley,et al.  Monte Carlo Methods , 1965 .

[162]  Thomas Kaminski,et al.  Recipes for adjoint code construction , 1998, TOMS.

[163]  N. Lind,et al.  Fast probability integration by three-parameter normal tail approximation , 1982 .

[164]  M. B. Beck,et al.  Water quality modeling: A review of the analysis of uncertainty , 1987 .

[165]  A. Owen.  Controlling correlations in latin hypercube samples , 1994 .

[166]  Eduard Hofer,et al.  Sensitivity analysis in the context of uncertainty analysis for computationally intensive models , 1999 .

[167]  J. Kleijnen.  Statistical tools for simulation practitioners , 1986 .

[168]  G. Yadigaroglu,et al.  On Farmer's line, probability density functions, and overall risk , 1986 .

[169]  N. R. Ortiz,et al.  Risk methodology for geologic disposal of radioactive waste: Final report , 1987 .

[170]  M S Chan,et al.  The consequences of uncertainty for the prediction of the effects of schistosomiasis control programmes , 1996, Epidemiology and Infection.

[171]  J. C. Helton,et al.  Statistical Analyses of Scatterplots to Identify Important Factors in Large-Scale Simulations, 1: Review and Comparison of Techniques , 1999 .

[172]  Yih-Tsuen Wu.  Demonstration of a new, fast probability integration method for reliability analysis , 1987 .

[173]  R Mead,et al.  A review of response surface methodology from a biometric viewpoint. , 1975, Biometrics.

[174]  N. Metropolis,et al.  The Monte Carlo method. , 1949 .

[175]  Laura Toran,et al.  Sensitivity analysis of solute transport in fractured porous media , 1995 .

[176]  E. Castillo,et al.  Uncertainty analyses in fault trees and Bayesian networks using FORM/SORM methods , 1999 .

[177]  R. C. MacDonald,et al.  Valuation of Supplemental and Enhanced Oil Recovery Projects With Risk Analysis , 1986 .

[178]  R. Iman,et al.  Rank correlation plots for use with correlated input variables , 1982 .

[179]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[180]  Simon Streltsov,et al.  Variance reduction algorithms for parallel replicated simulation of uniformized Markov chains , 1996, Discret. Event Dyn. Syst..

[181]  W. B. Murfin,et al.  The NUREG-1150 probabilistic risk assessment for the Sequoyah Nuclear Plant☆ , 1992 .

[182]  Norman R. Draper,et al.  An overview of design of experiments , 1996 .

[183]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[184]  A Cullen.  Addressing uncertainty-lessons from probabilistic exposure analysis. , 1999, Inhalation toxicology.

[185]  Alex H. Barbat,et al.  Monte Carlo techniques in computational stochastic mechanics , 1998 .

[186]  Jon C. Helton,et al.  Risk methodology for geologic disposal of radioactive waste: sensitivity analysis techniques , 1978 .

[187]  P. Holgate.  Tests of randomness based on distance methods , 1965 .

[188]  I. Sobol,et al.  About the use of rank transformation in sensitivity analysis of model output , 1995 .

[189]  E. Oblow,et al.  Sensitivity Theory for General Systems of Nonlinear Equations , 1980 .

[190]  R. Iman,et al.  A distribution-free approach to inducing rank correlation among input variables , 1982 .

[191]  Peter Eberhard,et al.  Automatic differentiation of numerical integration algorithms , 1999, Math. Comput..

[192]  Jerome Sacks,et al.  Designs for Computer Experiments , 1989 .

[193]  R. I. C. C. Francis,et al.  "Risk" in fisheries management: a review , 1997 .

[194]  J. Tukey.  THE PROPAGATION OF ERRORS, FLUCTUATIONS, AND TOLERANCES BASIC GENERALIZED FORMULAS , 1957 .

[195]  A. C. Payne,et al.  The NUREG-1150 probabilistic risk assessment for the Peach Bottom Atomic Power Station , 1992 .

[196]  R. A. Groeneveld,et al.  Practical Nonparametric Statistics (2nd ed). , 1981 .

[197]  B. Ripley.  Spatial Point Pattern Analysis in Ecology , 1987 .

[198]  Y. Ronen.  Uncertainty Analysis , 1988 .

[199]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[200]  Eugene P. Dougherty,et al.  A computational algorithm for the Green's function method of sensitivity analysis in chemical kinetics , 1979 .

[201]  I. Hasemann,et al.  Techniques applied in the COSYMA accident consequence uncertainty analysis , 2000 .

[202]  Kurt Binder,et al.  Applications of Monte Carlo methods to statistical physics , 1997 .

[203]  S. A. Cryer,et al.  Regional sensitivity analysis using a fractional factorial method for the USDA model GLEAMS , 1999, Environ. Model. Softw..

[204]  M. Elisabeth Paté-Cornell,et al.  Uncertainties in risk analysis: Six levels of treatment , 1996 .

[205]  William E. Kastenberg,et al.  A model for assessing and managing the risks of environmental lead emissions , 1999 .

[206]  D. Hamby.  A review of techniques for parameter sensitivity analysis of environmental models , 1994, Environmental monitoring and assessment.

[207]  G. Reinds,et al.  Uncertainty assessment in modelling soil acidification at the European scale : A case study , 1999 .

[208]  Wei-Liem Loh.  Estimating the integral of a squared regression function with Latin hypercube sampling , 1997 .

[209]  H Otway,et al.  Expert judgment in risk analysis and management: process, context, and pitfalls. , 1992, Risk analysis : an official publication of the Society for Risk Analysis.

[210]  Wallace B. Whiting,et al.  Effect of uncertainties in thermodynamic data and model parameters on calculated process performance , 1993 .

[211]  M. Jansen,et al.  Monte Carlo estimation of uncertainty contributions from several independent multivariate sources. , 1994 .

[212]  M. Jansen.  Analysis of variance designs for model output , 1999 .

[213]  E Ackerman,et al.  Parameter sensitivity of a model of viral epidemics simulated with Monte Carlo techniques. I. Illness attack rates. , 1993, International journal of bio-medical computing.

[214]  Guangzhou Zeng,et al.  A comparison of tests for randomness , 1985, Pattern Recognit..

[215]  Elizabeth J. Kelly,et al.  Separating Variability and Uncertainty in Environmental Risk Assessment—Making Choices , 2000 .

[216]  A. Lavenue,et al.  A modeling approach to address spatial variability within the Culebra Dolomite transmissivity field , 1992 .

[217]  Jon C. Helton,et al.  Representation of two-phase flow in the vicinity of the repository in the 1996 performance assessment for the Waste Isolation Pilot Plant , 2000, Reliab. Eng. Syst. Saf..

[218]  Peter N. Swift,et al.  Scenario development for the Waste Isolation Pilot Plant Compliance Certification Application , 2000, Reliab. Eng. Syst. Saf..

[219]  D. C. Cox,et al.  Limit lines for risk , 1982 .

[220]  A. Griewank,et al.  Automatic differentiation of algorithms : theory, implementation, and application , 1994 .

[221]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[222]  J S Evans,et al.  Use of probabilistic expert judgment in uncertainty analysis of carcinogenic potency. , 1994, Regulatory toxicology and pharmacology : RTP.

[223]  Ralph Haas,et al.  Sensitivity Analysis of Computer Models: World Bank HDM-III Model , 1999 .

[224]  D. Hamby.  A comparison of sensitivity analysis techniques. , 1995, Health physics.

[225]  Jon C. Helton,et al.  Uncertainty and sensitivity analysis for two-phase flow in the vicinity of the repository in the 1996 performance assessment for the Waste Isolation Pilot Plant: undisturbed conditions , 2000, Reliab. Eng. Syst. Saf..

[226]  Ronald L. Iman,et al.  Expert opinion in risk analysis: the NUREG-1150 methodology , 1989 .

[227]  G. J. Hahn,et al.  Statistical models in engineering , 1967 .

[228]  G. Apostolakis.  The concept of probability in safety assessments of technological systems. , 1990, Science.

[229]  Jon C. Helton,et al.  Stochastic and Subjective Uncertainty in the Assessment of Radiation Exposure at the Waste Isolation Pilot Plant , 1998 .

[230]  J. C. Loftis,et al.  Modeling Pesticide Transport in an Irrigated Field with Variable Water Application and Hydraulic Conductivity , 1998 .

[231]  Andreas Griewank,et al.  Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++ , 1996, TOMS.

[232]  D. Cacuci.  Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approach , 1981 .

[233]  Soon Heung Chang,et al.  A Study on Uncertainty and Sensitivity of Operational and Modelling Parameters for Feedwater Line Break Analysis , 1987 .

[234]  Ola Svenson,et al.  On expert judgements in safety analyses in the process industries , 1989 .

[235]  K. Shuler,et al.  Nonlinear sensitivity analysis of multiparameter model systems , 1977 .

[236]  M. Granger Morgan,et al.  Graphical Communication of Uncertain Quantities to Nontechnical People , 1987 .

[237]  T. Cruse,et al.  Advanced probabilistic structural analysis method for implicit performance functions , 1990 .

[238]  M. Eslami,et al.  Introduction to System Sensitivity Theory , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[239]  Enrique F. Castillo,et al.  Tail Uncertainty Analysis in Complex Systems , 1997, Artif. Intell..

[240]  Ranga Pitchumani,et al.  Stochastic modeling of nonisothermal flow during resin transfer molding , 1999 .

[241]  J. Peacock.  Two-dimensional goodness-of-fit testing in astronomy , 1983 .

[242]  Peter J. Diggle,et al.  On parameter estimation and goodness-of-fit testing for spatial point patterns , 1979 .

[243]  H. Rabitz,et al.  Efficient input-output model representations , 1999 .

[244]  F. Farmer.  REACTOR SAFETY AND SITING: A PROPOSED RISK CRITERION. , 1967 .

[245]  J. C. Helton,et al.  Uncertainty and sensitivity analysis in the presence of stochastic and subjective uncertainty , 1997 .

[246]  Ronald L. Iman,et al.  A FORTRAN-77 PROGRAM AND USER'S GUIDE FOR THE GENERATION OF LATIN HYPERCUBE AND RANDOM SAMPLES FOR USE WITH COMPUTER MODELS , 1984 .

[247]  F. Potra,et al.  Sensitivity analysis for atmospheric chemistry models via automatic differentiation , 1997 .

[248]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[249]  T. Turányi.  Sensitivity analysis of complex kinetic systems. Tools and applications , 1990 .

[250]  Stefano Tarantola,et al.  Sensitivity Analysis as an Ingredient of Modeling , 2000 .

[251]  A. Owen.  Lattice Sampling Revisited: Monte Carlo Variance of Means Over Randomized Orthogonal Arrays , 1994 .

[252]  A. Owen.  Monte Carlo Variance of Scrambled Net Quadrature , 1997 .

[253]  B. Ripley,et al.  On Sampling Spatial Patterns by Distance Methods , 1980 .

[254]  C. Allin Cornell,et al.  Use of Technical Expert Panels: Applications to Probabilistic Seismic Hazard Analysis * , 1998 .

[255]  Jerry Banks,et al.  Handbook of simulation - principles, methodology, advances, applications, and practice , 1998, A Wiley-Interscience publication.

[256]  F. James,et al.  Monte Carlo theory and practice , 1980 .

[257]  Christian Bischof,et al.  Adifor 2.0: automatic differentiation of Fortran 77 programs , 1996 .

[258]  M. G. Marietta,et al.  Uncertainty and sensitivity analysis results obtained in the 1992 performance assessment for the waste isolation pilot plant , 1996 .

[259]  J. Besag,et al.  Statistical Analysis of Spatial Point Patterns by Means of Distance Methods , 1976 .

[260]  H J Clewell,et al.  Investigation of the impact of pharmacokinetic variability and uncertainty on risks predicted with a pharmacokinetic model for chloroform. , 1996, Toxicology.

[261]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[262]  R. O. Gilbert,et al.  On the use of uncertainty analyses to test hypotheses regarding deterministic model predictions of environmental processes , 1995 .

[263]  S. R. Jammalamadaka,et al.  Against the Gods: The Remarkable Story of Risk , 1999 .

[264]  G. W. Parry,et al.  Characterization and evaluation of uncertainty in probabilistic risk analysis , 1981 .

[265]  R. Iman,et al.  The Use of the Rank Transform in Regression , 1979 .

[266]  Ian Hacking,et al.  The Emergence of Probability. A Philosophical Study of Early Ideas about Probability, Induction and Statistical Inference , 1979 .

[267]  J. Turner,et al.  A Monte Carlo primer for health physicists. , 1985, Health physics.

[268]  J. C. Helton,et al.  An Investigation of Uncertainty and Sensitivity Analysis Techniques for Computer Models , 1988 .

[269]  Jon C. Helton,et al.  Radionuclide transport in the vicinity of the repository and associated complementary cumulative distribution functions in the 1996 performance assessment for the Waste Isolation Pilot Plant , 2000, Reliab. Eng. Syst. Saf..

[270]  Jon C. Helton,et al.  An Approach to Sensitivity Analysis of Computer Models: Part II - Ranking of Input Variables, Response Surface Validation, Distribution Effect and Technique Synopsis , 1981 .

[271]  Jon C. Helton,et al.  Summary description of the methods used in the probabilistic risk assessments for NUREG-1150 , 1992 .

[272]  H. Gershengorn,et al.  A tale of two futures: HIV and antiretroviral therapy in San Francisco. , 2000, Science.

[273]  Marc Soutter,et al.  Global sensitivity analyses of three pesticide leaching models using a Monte-Carlo approach , 1999 .

[274]  T. Barry.  Recommendations on the Testing and Use of Pseudo‐Random Number Generators Used in Monte Carlo Analysis for Risk Assessment , 1995 .