Identification and Review of Sensitivity Analysis Methods

Identification and qualitative comparison of sensitivity analysis methods that have been used across various disciplines, and that merit consideration for application to food‐safety risk assessment models, are presented in this article. Sensitivity analysis can help in identifying critical control points, prioritizing additional data collection or research, and verifying and validating a model. Ten sensitivity analysis methods, including four mathematical methods, five statistical methods, and one graphical method, are identified. The selected methods are compared on the basis of their applicability to different types of models, computational issues such as initial data requirement and complexity of their application, representation of the sensitivity, and the specific uses of these methods. Applications of these methods are illustrated with examples from various fields. No one method is clearly best for food‐safety risk models. In general, use of two or more methods, preferably with dissimilar theoretical foundations, may be needed to increase confidence in the ranking of key inputs.

[1]  Luiz Paulo Fávero,et al.  Design and Analysis of Experiments , 2001, Handbook of statistics.

[2]  David E. Booth,et al.  Applied Multivariate Analysis , 2003, Technometrics.

[3]  U. Ligges Review of An R and S-PLUS companion to applied regression by J. Fox, Sage Publications, Thousand Oaks, California 2002 , 2003 .

[4]  Paul D. Minton,et al.  Statistics: The Exploration and Analysis of Data , 2002, Technometrics.

[5]  Joseph D. Conklin Applied Logistic Regression , 2002, Technometrics.

[6]  Sitakanta Mohanty,et al.  Sensitivity analysis of a complex, proposed geologic waste disposal system using the Fourier Amplitude Sensitivity Test method , 2001, Reliab. Eng. Syst. Saf..

[7]  H. Künsch,et al.  Practical identifiability analysis of large environmental simulation models , 2001 .

[8]  P Rossier,et al.  An initial investigation of the reliability of the Rivermead Extended ADL index in patients presenting with neurological impairment. , 2001, Journal of rehabilitation medicine.

[9]  M. A. Pizarro,et al.  Variations in Reflectance of Tropical Soils: Spectral-Chemical Composition Relationships from AVIRIS data , 2001 .

[10]  Otto Rentz,et al.  Integrated technique assessment with imprecise information as a support for the identification of best available techniques (BAT) , 2001, OR Spectr..

[11]  D. Vose Risk Analysis: A Quantitative Guide , 2000 .

[12]  D. A. Ratkowsky,et al.  Growth Limits of Listeria monocytogenesas a Function of Temperature, pH, NaCl, and Lactic Acid , 2000, Applied and Environmental Microbiology.

[13]  D G Altman,et al.  Indirect comparison in evaluating relative efficacy illustrated by antimicrobial prophylaxis in colorectal surgery. , 2000, Controlled clinical trials.

[14]  A. Phillips,et al.  Multiple sequence alignment in phylogenetic analysis. , 2000, Molecular phylogenetics and evolution.

[15]  J. L. Smith,et al.  Microbial risk assessment: dose-response relations and risk characterization. , 2000, International journal of food microbiology.

[16]  D Fraedrich,et al.  A methodological framework for the validation of predictive simulations , 2000, Eur. J. Oper. Res..

[17]  John A. Reed,et al.  Computational Simulation of Gas Turbines: Part 1—Foundations of Component-Based Models , 2000 .

[18]  Sven P. Jacobsson,et al.  Algorithmic approaches for studies of variable influence, contribution and selection in neural networks , 2000 .

[19]  Robert H. Kewley,et al.  Data strip mining for the virtual design of pharmaceuticals with neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[20]  E. Deconinck,et al.  Cost-effectiveness of CD34+ dose in peripheral blood progenitor cell transplantation for non-Hodgkin’s lymphoma patients: a single centre study , 2000, Bone Marrow Transplantation.

[21]  David Sartori,et al.  Experimental Design Techniquesin Statistical Practice: A Practical Software-Based Approach , 2000, Technometrics.

[22]  Roger Jones,et al.  Analysing the risk of climate change using an irrigation demand model , 2000 .

[23]  L Kreienbrock,et al.  Models for retrospective quantification of indoor radon exposure in case-control studies. , 2000, Health physics.

[24]  P M Bongers,et al.  Physical risk factors for neck pain. , 2000, Scandinavian journal of work, environment & health.

[25]  L. Jaykus,et al.  Validation and analysis of modeled predictions of growth of Bacillus cereus spores in boiled rice. , 2000, Journal of food protection.

[26]  Byung Hwan Oh,et al.  Sensitivity Analysis of Time-Dependent Behavior in PSC Box Girder Bridges , 2000 .

[27]  Kody Varahramyan,et al.  Statistical optimization and manufacturing sensitivity analysis of 0.18 mm SOI MOSFETs , 1999 .

[28]  J J Pet-Armacost,et al.  Monte Carlo Sensitivity Analysis of Unknown Parameters in Hazardous Materials Transportation Risk Assessment , 1999, Risk analysis : an official publication of the Society for Risk Analysis.

[29]  H. R. Ashraf,et al.  Microbiological survey of tofu sold in a rural Illinois county. , 1999, Journal of food protection.

[30]  E. Monteleone,et al.  Methodological approach to evaluate the effects of age at slaughter and storage temperature and time on sensory profile of lamb meat. , 1999, Meat science.

[31]  Jon C. Helton,et al.  A FORTRAN 77 Program and User's Guide for the Statistical Analyses of Scatterplots to Identify Important Factors in Large-Scale Simulations , 1999 .

[32]  Mitchell J. Small,et al.  Expert System Methodology for Evaluating Reductive Dechlorination at TCE Sites , 1999 .

[33]  Scott Baker,et al.  Survey of Risk Management in Major U.K. Companies , 1999 .

[34]  L. McCaig,et al.  Food-related illness and death in the United States. , 1999, Emerging infectious diseases.

[35]  Bernt J. Leira,et al.  Reliability-based analysis of a stretch-bending process for aluminium extrusions , 1999 .

[36]  Stefano Tarantola,et al.  A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output , 1999, Technometrics.

[37]  K Fujimoto,et al.  Correlation between indoor radon concentration and dose rate in air from terrestrial gamma radiation in Japan. , 1998, Health physics.

[38]  H M Marks,et al.  Topics in Microbial Risk Assessment: Dynamic Flow Tree Process , 1998, Risk analysis : an official publication of the Society for Risk Analysis.

[39]  Roni Avissar,et al.  Comparison of three land‐surface schemes with the Fourier amplitude sensitivity test (FAST) , 1998 .

[40]  L. Manheim,et al.  Health services research clinical trials: issues in the evaluation of economic costs and benefits. , 1998, Controlled clinical trials.

[41]  A. Saltelli,et al.  An alternative way to compute Fourier amplitude sensitivity test (FAST) , 1998 .

[42]  Anna M. Lammerding,et al.  An Overview of Microbial Food Safety Risk Assessment †. , 1997, Journal of food protection.

[43]  Eric P. Kvam,et al.  Application of analysis of variance (ANOVA) statistical methods to breaking load corrosion test , 1997 .

[44]  R. L. Buchanan,et al.  Identifying and controlling emerging foodborne pathogens: research needs. , 1997, Emerging infectious diseases.

[45]  M. Ariel,et al.  Membrane properties and monosynaptic retinal excitation of neurons in the turtle accessory optic system. , 1997, Journal of neurophysiology.

[46]  G. Wotawa,et al.  Estimating the uncertainty of a Lagrangian photochemical air quality simulation model caused by inexact meteorological input data , 1997 .

[47]  M. B. Beck,et al.  On the problem of model validation for predictive exposure assessments , 1997 .

[48]  Digvir S. Jayas,et al.  Sensitivity analysis of a mathematical model to simulate aeration of wheat stored in Brazil , 1997 .

[49]  Howard R. Moskowitz,et al.  A commercial application of RSM for ready to eat cereal , 1997 .

[50]  R. Fricker Statistical Problem Solving in Quality Engineering , 1997 .

[51]  Thomas A. Fontaine,et al.  SENSITWITY ANALYSIS OF SIMULATED CONTAMINATED SEDIMENT TRANSPORT 1 , 1997 .

[52]  Dongming Hwang,et al.  An automatic differentiation technique for sensitivity analysis of numerical advection schemes in air quality models , 1997 .

[53]  Rakesh K. Kapania,et al.  Aeroelastic Sensitivity Analysis of Wings , 1997 .

[54]  Norman R. Draper,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1997 .

[55]  F. Potra,et al.  Sensitivity analysis for atmospheric chemistry models via automatic differentiation , 1997 .

[56]  S. Menard Applied Logistic Regression Analysis , 1996 .

[57]  R. Weiss An approach to Bayesian sensitivity analysis , 1996 .

[58]  Moon Hyun Chun,et al.  A combined procedure of RSM and LHS for uncertainty analyses in source term quantifications using MAAP3.0B code , 1996 .

[59]  Christian Bischof,et al.  Adifor 2.0: automatic differentiation of Fortran 77 programs , 1996 .

[60]  Tim E. Carpenter,et al.  Simulation modeling of the effect of climatic factors on bluetongue virus infection in Australian cattle herds. I. model formulation, verification and validation , 1996 .

[61]  Thomas E. McKone,et al.  Overview of the risk analysis approach and terminology: the merging of science, judgment and values , 1996 .

[62]  Joseph W. McKean,et al.  A Monte Carlo study of robust and least squares response surface methods , 1996 .

[63]  Roni Avissar,et al.  Sensitivity of shallow convective precipitation induced by land surface heterogeneities to dynamical and cloud microphysical parameters , 1996 .

[64]  S. Fleming,et al.  Breakeven under capitation: Pure and simple? , 1996, Health care management review.

[65]  Sabine Glaser,et al.  Application of Statistical Classification Methods to the Evaluation of In Vitro Tests , 1996 .

[66]  A. Goldman An Introduction to Regression Graphics , 1995 .

[67]  Martin Berz,et al.  Higher-order sensitivity analysis of finite element method by automatic differentiation , 1995 .

[68]  Thomas J. Kazmierski Statistical Problem Solving in Quality Engineering , 1995 .

[69]  J. Kleijnen Verification and validation of simulation models , 1995 .

[70]  Marcel H. Zwietering,et al.  The HACCP concept: specification of criteria using quantitative risk assessment , 1995 .

[71]  Robert L. Buchanan,et al.  The role of microbiological criteria and risk assessment in HACCP , 1995 .

[72]  Mitchell J. Small,et al.  Risk‐based environmental remediation: Decision framework and role of uncertainty , 1994 .

[73]  Roni Avissar,et al.  An Evaluation with the Fourier Amplitude Sensitivity Test (FAST) of Which Land-Surface Parameters Are of Greatest Importance in Atmospheric Modeling , 1994 .

[74]  Carl R. Dillon,et al.  Advanced breakeven analysis of agricultural enterprise budgets , 1993 .

[75]  C B Ernhart,et al.  Dentine lead and intelligence prior to school entry: a statistical sensitivity analysis. , 1993, Journal of clinical epidemiology.

[76]  N. T. Hagen Macroparasitic epizootic disease: a potential mechanism for the termination of sea urchin outbreaks in Northern Norway? , 1992 .

[77]  Gary E. Blau,et al.  The Role of Sensitivity Analysis in Groundwater Risk Modeling for Pesticides , 1992, Weed Technology.

[78]  J. H. Schuenemeyer,et al.  Generalized Linear Models (2nd ed.) , 1992 .

[79]  J. Stott,et al.  Alteration in lymphocyte subpopulations in bovine leukosis virus-infected cattle. , 1992, Veterinary immunology and immunopathology.

[80]  Muni S. Srivastava,et al.  Regression Analysis: Theory, Methods, and Applications , 1991 .

[81]  T.C.E. Cheng,et al.  EPQ with Process Capability and Quality Assurance Considerations , 1991 .

[82]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[83]  K E Willard,et al.  Probabilistic Analysis of Decision Trees Using Monte Carlo Simulation , 1986, Medical decision making : an international journal of the Society for Medical Decision Making.

[84]  K E Willard,et al.  Probabilistic Analysis of Decision Trees Using Symbolic Algebra , 1986, Medical decision making : an international journal of the Society for Medical Decision Making.

[85]  Dennis Doverspike,et al.  Frequently Encountered Problems in the Application of Regression Analysis to the Investigation of Sex Discrimination in Salaries , 1986 .

[86]  Ronald L. Iman,et al.  FORTRAN 77 program and user's guide for the calculation of partial correlation and standardized regression coefficients , 1985 .

[87]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[88]  Division on Earth Risk Assessment in the Federal Government: Managing the Process , 1983 .

[89]  Herbert A. Simon,et al.  The Sciences of the Artificial - 3rd Edition , 1981 .

[90]  S. Kaplan,et al.  On The Quantitative Definition of Risk , 1981 .

[91]  Gershon Kedem,et al.  Automatic Differentiation of Computer Programs , 1980, TOMS.

[92]  Jae-On Kim,et al.  Introduction to Factor Analysis: What It Is and How To Do It , 1978 .

[93]  Hon-Shiang Lau,et al.  Stochastic Breakeven Analysis , 1978 .

[94]  Tony Greenfield,et al.  Probability and Statistics for Engineers and Scientists. , 1978 .

[95]  K.,et al.  Nonlinear sensitivity analysis of multiparameter model systems , 1977 .

[96]  Charles S. Tapiero,et al.  Linear Breakeven Analysis Under Risk , 1975 .

[97]  K. Shuler,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. III. Analysis of the approximations , 1975 .

[98]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[99]  C. Fortuin,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory , 1973 .

[100]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[101]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[102]  Seymour Geisser,et al.  Statistical Principles in Experimental Design , 1963 .

[103]  Walter L. Smith Probability and Statistics , 1959, Nature.

[104]  Maurice G. Kendall,et al.  Rank Correlation Methods , 1949 .

[105]  Debbie L. Hahs-Vaughn,et al.  Multiple Comparison Procedures , 2013 .

[106]  F. Kuchler Valuing the Health Benefits of Food Safety: A Proceedings , 2001 .

[107]  Jack P. C. Kleijnen,et al.  A methodology for fitting and validating metamodels in simulation , 2000, Eur. J. Oper. Res..

[108]  Shiv O. Prasher,et al.  PREDICTION OF ANNUAL NITRATE-N LOSSES IN DRAIN OUTFLOWS WITH ARTIFICIAL NEURAL NETWORKS , 2000 .

[109]  S J van Gerwen,et al.  Sensitivity analysis in quantitative microbial risk assessment. , 2000, International journal of food microbiology.

[110]  J. C. Heltona,et al.  Uncertainty and Sensitivity Analysis for Two-Phase the Vicinity of the Repository in the d 996 Performance Assessment for the Waste Isolation Pilot Plant : Disturbed Conditions , 2000 .

[111]  B. Christianson Cheap Newton steps for optimal control problems: automatic differentiation and Pantoja's algorithm , 1999 .

[112]  H. Christopher Frey,et al.  Desktop Modeling of the Performance, Emissions, and Cost of Gasification Systems , 1998 .

[113]  Joyce Snell,et al.  6. Alternative Methods of Regression , 1996 .

[114]  John W. Hall,et al.  Development and evaluation of multiple regression models for prediction of sweet cherry liking , 1995 .

[115]  Scott Ferson,et al.  Sensitivity analysis for models of population viability , 1995 .

[116]  J. C. Helton,et al.  Uncertainty and sensitivity analysis of food pathway results with the MACCS Reactor Accident Consequence Model , 1995 .

[117]  Steve Hathaway,et al.  Harmonization of international requirements under HACCP-based food control systems , 1995 .

[118]  A. Di Rienzo,et al.  Diagnosis of acute myeloid leukemia and system Coulter VCS. , 1994, Haematologica.

[119]  Jon C. Helton,et al.  Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal , 1993 .

[120]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[121]  Jon C. Helton,et al.  Calculation of reactor accident safety goals , 1993 .

[122]  C. Bischof,et al.  ADIFOR-Generating Derivative Codes from Fortran Programs , 1992 .

[123]  Charalampos Baniotopoulos,et al.  A contribution to the sensitivity analysis of the sea-bed-structure interaction problem for underwater pipelines , 1991 .

[124]  M. G. Morgan,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[125]  S A Maisto,et al.  Comparison of two techniques to obtain retrospective reports of drinking behavior from alcohol abusers. , 1982, Addictive behaviors.

[126]  John H. Seinfeld,et al.  Global sensitivity analysis—a computational implementation of the Fourier Amplitude Sensitivity Test (FAST) , 1982 .

[127]  L. B. Rall Applications of Software for Automatic Differentiation in Numerical Computation , 1980 .

[128]  H. Lindman Analysis of variance in complex experimental designs , 1974 .

[129]  D. M. Ellis,et al.  Applied Regression Analysis , 1968 .