OF SENSITIVITY ANALYSIS

Identification and qualitative comparison of sensitivity analysis methods that have been used across various disciplines, and that merit consideration for application to food-safety risk assessment models, are presented in this article. Sensitivity analysis can help in identifying critical control points, prioritizing additional data collection or research, and verifying and validating a model. Ten sensitivity analysis methods, including four mathematical methods, five statistical methods, and one graphical method, are identified. The selected methods are compared on the basis of their applicability to different types of models, computational issues such as initial data requirement and complexity of their application, representation of the sensitivity, and the specific uses of these methods. Applications of these methods are illustrated with examples from various fields. No one method is clearly best for food-safety risk models. In general, use of two or more methods, preferably with dissimilar theoretical foundations, may be needed to increase confidence in the ranking of key inputs.

[1]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[2]  P Grandjean,et al.  Dentine lead and intelligence prior to school entry: a statistical sensitivity analysis. , 1993, Journal of clinical epidemiology.

[3]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[4]  Tim E. Carpenter,et al.  Simulation modeling of the effect of climatic factors on bluetongue virus infection in Australian cattle herds. I. model formulation, verification and validation , 1996 .

[5]  Division on Earth Risk Assessment in the Federal Government: Managing the Process , 1983 .

[6]  John H. Seinfeld,et al.  Global sensitivity analysis—a computational implementation of the Fourier Amplitude Sensitivity Test (FAST) , 1982 .

[7]  P M Bongers,et al.  Physical risk factors for neck pain. , 2000, Scandinavian journal of work, environment & health.

[8]  T.C.E. Cheng,et al.  EPQ with Process Capability and Quality Assurance Considerations , 1991 .

[9]  U. Ligges Review of An R and S-PLUS companion to applied regression by J. Fox, Sage Publications, Thousand Oaks, California 2002 , 2003 .

[10]  Gary E. Blau,et al.  The Role of Sensitivity Analysis in Groundwater Risk Modeling for Pesticides , 1992, Weed Technology.

[11]  Jon C. Helton,et al.  A FORTRAN 77 Program and User's Guide for the Statistical Analyses of Scatterplots to Identify Important Factors in Large-Scale Simulations , 1999 .

[12]  Moon Hyun Chun,et al.  A combined procedure of RSM and LHS for uncertainty analyses in source term quantifications using MAAP3.0B code , 1996 .

[13]  H. R. Ashraf,et al.  Microbiological survey of tofu sold in a rural Illinois county. , 1999, Journal of food protection.

[14]  S. Menard Applied Logistic Regression Analysis , 1996 .

[15]  John W. Hall,et al.  Development and evaluation of multiple regression models for prediction of sweet cherry liking , 1995 .

[16]  S. Fleming,et al.  Breakeven under capitation: Pure and simple? , 1996, Health care management review.

[17]  J J Pet-Armacost,et al.  Monte Carlo Sensitivity Analysis of Unknown Parameters in Hazardous Materials Transportation Risk Assessment , 1999, Risk analysis : an official publication of the Society for Risk Analysis.

[18]  Christian Bischof,et al.  Adifor 2.0: automatic differentiation of Fortran 77 programs , 1996 .

[19]  Michael H. Kutner Applied Linear Statistical Models , 1974 .

[20]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[21]  F. Potra,et al.  Sensitivity analysis for atmospheric chemistry models via automatic differentiation , 1997 .

[22]  Scott Baker,et al.  Survey of Risk Management in Major U.K. Companies , 1999 .

[23]  Roger Jones,et al.  Analysing the risk of climate change using an irrigation demand model , 2000 .

[24]  M. B. Beck,et al.  On the problem of model validation for predictive exposure assessments , 1997 .

[25]  S. Kaplan,et al.  On The Quantitative Definition of Risk , 1981 .

[26]  P Rossier,et al.  An initial investigation of the reliability of the Rivermead Extended ADL index in patients presenting with neurological impairment. , 2001, Journal of rehabilitation medicine.

[27]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[28]  Joseph W. McKean,et al.  A Monte Carlo study of robust and least squares response surface methods , 1996 .

[29]  M. Kendall Rank Correlation Methods , 1949 .

[30]  Muni S. Srivastava,et al.  Regression Analysis: Theory, Methods, and Applications , 1991 .

[31]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[32]  Paul D. Minton,et al.  Statistics: The Exploration and Analysis of Data , 2002, Technometrics.

[33]  E. Deconinck,et al.  Cost-effectiveness of CD34+ dose in peripheral blood progenitor cell transplantation for non-Hodgkin’s lymphoma patients: a single centre study , 2000, Bone Marrow Transplantation.

[34]  K E Willard,et al.  Probabilistic Analysis of Decision Trees Using Monte Carlo Simulation , 1986, Medical decision making : an international journal of the Society for Medical Decision Making.

[35]  C. Hedberg,et al.  Food-related illness and death in the United States. , 1999, Emerging infectious diseases.

[36]  Byung Hwan Oh,et al.  Sensitivity Analysis of Time-Dependent Behavior in PSC Box Girder Bridges , 2000 .

[37]  B. Christianson Cheap Newton steps for optimal control problems: automatic differentiation and Pantoja's algorithm , 1999 .

[38]  Marcel H. Zwietering,et al.  The HACCP concept: specification of criteria using quantitative risk assessment , 1995 .

[39]  N. T. Hagen Macroparasitic epizootic disease: a potential mechanism for the termination of sea urchin outbreaks in Northern Norway? , 1992 .

[40]  Roni Avissar,et al.  Comparison of three land‐surface schemes with the Fourier amplitude sensitivity test (FAST) , 1998 .

[41]  Otto Rentz,et al.  Integrated technique assessment with imprecise information as a support for the identification of best available techniques (BAT) , 2001, OR Spectr..

[42]  Robert L. Buchanan,et al.  The role of microbiological criteria and risk assessment in HACCP , 1995 .

[43]  Digvir S. Jayas,et al.  Sensitivity analysis of a mathematical model to simulate aeration of wheat stored in Brazil , 1997 .

[44]  John A. Reed,et al.  Computational Simulation of Gas Turbines: Part 1—Foundations of Component-Based Models , 2000 .

[45]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[46]  Eric P. Kvam,et al.  Application of analysis of variance (ANOVA) statistical methods to breaking load corrosion test , 1997 .

[47]  Jack P. C. Kleijnen,et al.  EUROPEAN JOURNAL OF OPERATIONAL , 1992 .

[48]  A. Phillips,et al.  Multiple sequence alignment in phylogenetic analysis. , 2000, Molecular phylogenetics and evolution.

[49]  R. Weiss An approach to Bayesian sensitivity analysis , 1996 .

[50]  K Fujimoto,et al.  Correlation between indoor radon concentration and dose rate in air from terrestrial gamma radiation in Japan. , 1998, Health physics.

[51]  Carl R. Dillon,et al.  Advanced breakeven analysis of agricultural enterprise budgets , 1993 .

[52]  L Kreienbrock,et al.  Models for retrospective quantification of indoor radon exposure in case-control studies. , 2000, Health physics.

[53]  E. Monteleone,et al.  Methodological approach to evaluate the effects of age at slaughter and storage temperature and time on sensory profile of lamb meat. , 1999, Meat science.

[54]  David E. Booth,et al.  Applied Multivariate Analysis , 2003, Technometrics.

[55]  B. J. Winer,et al.  Statistical Principles in Experimental Design, 2nd Edition. , 1973 .

[56]  S J van Gerwen,et al.  Sensitivity analysis in quantitative microbial risk assessment. , 2000, International journal of food microbiology.

[57]  H. Lindman Analysis of variance in complex experimental designs , 1974 .

[58]  CarleAlan,et al.  ADIFOR-Generating Derivative Codes from Fortran Programs , 1992 .

[59]  Kody Varahramyan,et al.  Statistical optimization and manufacturing sensitivity analysis of 0.18 mm SOI MOSFETs , 1999 .

[60]  A. Saltelli,et al.  An alternative way to compute Fourier amplitude sensitivity test (FAST) , 1998 .

[61]  Jae-On Kim,et al.  Introduction to Factor Analysis: What It Is and How To Do It , 1978 .

[62]  D Fraedrich,et al.  A methodological framework for the validation of predictive simulations , 2000, Eur. J. Oper. Res..

[63]  Jon C. Helton,et al.  Calculation of reactor accident safety goals , 1993 .

[64]  S A Maisto,et al.  Comparison of two techniques to obtain retrospective reports of drinking behavior from alcohol abusers. , 1982, Addictive behaviors.

[65]  Charalampos Baniotopoulos,et al.  A contribution to the sensitivity analysis of the sea-bed-structure interaction problem for underwater pipelines , 1991 .

[66]  Dongming Hwang,et al.  An automatic differentiation technique for sensitivity analysis of numerical advection schemes in air quality models , 1997 .

[67]  Bernt J. Leira,et al.  Reliability-based analysis of a stretch-bending process for aluminium extrusions , 1999 .

[68]  J. Stott,et al.  Alteration in lymphocyte subpopulations in bovine leukosis virus-infected cattle. , 1992, Veterinary immunology and immunopathology.

[69]  Shiv O. Prasher,et al.  PREDICTION OF ANNUAL NITRATE-N LOSSES IN DRAIN OUTFLOWS WITH ARTIFICIAL NEURAL NETWORKS , 2000 .

[70]  D G Altman,et al.  Indirect comparison in evaluating relative efficacy illustrated by antimicrobial prophylaxis in colorectal surgery. , 2000, Controlled clinical trials.

[71]  Mitchell J. Small,et al.  Expert System Methodology for Evaluating Reductive Dechlorination at TCE Sites , 1999 .

[72]  Steve Hathaway,et al.  Harmonization of international requirements under HACCP-based food control systems , 1995 .

[73]  Jon C. Helton,et al.  Uncertainty and sensitivity analysis for two-phase flow in the vicinity of the repository in the 1996 performance assessment for the Waste Isolation Pilot Plant: undisturbed conditions , 2000, Reliab. Eng. Syst. Saf..

[74]  F. Kuchler Valuing the Health Benefits of Food Safety: A Proceedings , 2001 .

[75]  Howard R. Moskowitz,et al.  A commercial application of RSM for ready to eat cereal , 1997 .

[76]  Dennis Doverspike,et al.  Frequently Encountered Problems in the Application of Regression Analysis to the Investigation of Sex Discrimination in Salaries , 1986 .

[77]  Anna M. Lammerding,et al.  An Overview of Microbial Food Safety Risk Assessment †. , 1997, Journal of food protection.

[78]  Jon C. Helton,et al.  Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal , 1993 .

[79]  H. Christopher Frey,et al.  Desktop Modeling of the Performance, Emissions, and Cost of Gasification Systems , 1998 .

[80]  D. A. Ratkowsky,et al.  Growth Limits of Listeria monocytogenesas a Function of Temperature, pH, NaCl, and Lactic Acid , 2000, Applied and Environmental Microbiology.

[81]  Sitakanta Mohanty,et al.  Sensitivity analysis of a complex, proposed geologic waste disposal system using the Fourier Amplitude Sensitivity Test method , 2001, Reliab. Eng. Syst. Saf..

[82]  R. L. Buchanan,et al.  Identifying and controlling emerging foodborne pathogens: research needs. , 1997, Emerging infectious diseases.

[83]  K E Willard,et al.  Probabilistic Analysis of Decision Trees Using Symbolic Algebra , 1986, Medical decision making : an international journal of the Society for Medical Decision Making.

[84]  J. L. Smith,et al.  Microbial risk assessment: dose-response relations and risk characterization. , 2000, International journal of food microbiology.

[85]  R. H. Myers,et al.  Probability and Statistics for Engineers and Scientists , 1978 .

[86]  Roni Avissar,et al.  Sensitivity of shallow convective precipitation induced by land surface heterogeneities to dynamical and cloud microphysical parameters , 1996 .

[87]  Mitchell J. Small,et al.  Risk‐based environmental remediation: Decision framework and role of uncertainty , 1994 .

[88]  Sven P. Jacobsson,et al.  Algorithmic approaches for studies of variable influence, contribution and selection in neural networks , 2000 .

[89]  Thomas E. McKone,et al.  Overview of the risk analysis approach and terminology: the merging of science, judgment and values , 1996 .

[90]  Jack P. C. Kleijnen,et al.  A methodology for fitting and validating metamodels in simulation , 2000, Eur. J. Oper. Res..

[91]  Thomas A. Fontaine,et al.  SENSITWITY ANALYSIS OF SIMULATED CONTAMINATED SEDIMENT TRANSPORT 1 , 1997 .

[92]  M. Ariel,et al.  Membrane properties and monosynaptic retinal excitation of neurons in the turtle accessory optic system. , 1997, Journal of neurophysiology.

[93]  A. Di Rienzo,et al.  Diagnosis of acute myeloid leukemia and system Coulter VCS. , 1994, Haematologica.

[94]  H. Künsch,et al.  Practical identifiability analysis of large environmental simulation models , 2001 .

[95]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[96]  K.,et al.  Nonlinear sensitivity analysis of multiparameter model systems , 1977 .

[97]  Joyce Snell,et al.  6. Alternative Methods of Regression , 1996 .

[98]  G. Wotawa,et al.  Estimating the uncertainty of a Lagrangian photochemical air quality simulation model caused by inexact meteorological input data , 1997 .

[99]  Thomas J. Kazmierski Statistical Problem Solving in Quality Engineering , 1995 .

[100]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[101]  Joseph A. C. Delaney Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.

[102]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[103]  Robert H. Kewley,et al.  Data strip mining for the virtual design of pharmaceuticals with neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[104]  Robert L. Mason,et al.  Statistical Principles in Experimental Design , 2003 .

[105]  C. Fortuin,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory , 1973 .

[106]  Hon-Shiang Lau,et al.  Stochastic Breakeven Analysis , 1978 .

[107]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[108]  Scott Ferson,et al.  Sensitivity analysis for models of population viability , 1995 .

[109]  H M Marks,et al.  Topics in Microbial Risk Assessment: Dynamic Flow Tree Process , 1998, Risk analysis : an official publication of the Society for Risk Analysis.

[110]  Charles S. Tapiero,et al.  Linear Breakeven Analysis Under Risk , 1975 .

[111]  M. A. Pizarro,et al.  Variations in Reflectance of Tropical Soils: Spectral-Chemical Composition Relationships from AVIRIS data , 2001 .

[112]  Sabine Glaser,et al.  Application of Statistical Classification Methods to the Evaluation of In Vitro Tests , 1996 .

[113]  A. Goldman An Introduction to Regression Graphics , 1995 .

[114]  Stefano Tarantola,et al.  A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output , 1999, Technometrics.

[115]  L. B. Rall Applications of Software for Automatic Differentiation in Numerical Computation , 1980 .

[116]  K. Shuler,et al.  Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. III. Analysis of the approximations , 1975 .

[117]  Ronald L. Iman,et al.  FORTRAN 77 program and user's guide for the calculation of partial correlation and standardized regression coefficients , 1985 .

[118]  L. Manheim,et al.  Health services research clinical trials: issues in the evaluation of economic costs and benefits. , 1998, Controlled clinical trials.

[119]  Jon C. Helton,et al.  Uncertainty and sensitivity analysis of food pathway results with the MACCS Reactor Accident Consequence Model , 1995 .

[120]  B. J. Winer Statistical Principles in Experimental Design , 1992 .

[121]  L. Jaykus,et al.  Validation and analysis of modeled predictions of growth of Bacillus cereus spores in boiled rice. , 2000, Journal of food protection.

[122]  Roni Avissar,et al.  An Evaluation with the Fourier Amplitude Sensitivity Test (FAST) of Which Land-Surface Parameters Are of Greatest Importance in Atmospheric Modeling , 1994 .

[123]  Gershon Kedem,et al.  Automatic Differentiation of Computer Programs , 1980, TOMS.

[124]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[125]  Rakesh K. Kapania,et al.  Aeroelastic Sensitivity Analysis of Wings , 1997 .

[126]  Martin Berz,et al.  Higher-order sensitivity analysis of finite element method by automatic differentiation , 1995 .

[127]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[128]  David Sartori,et al.  Experimental Design Techniquesin Statistical Practice: A Practical Software-Based Approach , 2000, Technometrics.

[129]  Herbert A. Simon,et al.  The Sciences of the Artificial - 3rd Edition , 1981 .

[130]  C B Ernhart,et al.  Dentine lead and intelligence prior to school entry: a statistical sensitivity analysis. , 1993, Journal of clinical epidemiology.

[131]  H. Simon The Sciences of the Artificial, (Third edition) , 1997 .