Evaluation of Sampling-Based Methods for Sensitivity Analysis: Case Study for the E. coli Food Safety Process Risk Model

ABSTRACT This article evaluates selected sensitivity analysis methods applicable to risk assessment models with two-dimensional probabilistic frameworks, using a microbial food safety process risk model as a test-bed. Six sampling-based sensitivity analysis methods were evaluated including Pearson and Spearman correlation, sample and rank linear regression, and sample and rank stepwise regression. In a two-dimensional risk model, the identification of key controllable inputs that can be priorities for risk management can be confounded by uncertainty. However, despite uncertainty, results show that key inputs can be distinguished from those that are unimportant, and inputs can be grouped into categories of similar levels of importance. All selected methods are capable of identifying unimportant inputs, which is helpful in that efforts to collect data to improve the assessment or to focus risk management strategies can be prioritized elsewhere. Rank-based methods provided more robust insights with respect to the key sources of variability in that they produced narrower ranges of uncertainty for sensitivity results and more clear distinctions when comparing the importance of inputs or groups of inputs. Regression-based methods have advantages over correlation approaches because they can be configured to provide insight regarding interactions and nonlinearities in the model.

[1]  David V. Hinkley,et al.  Inference about the change-point in a sequence of binomial variables , 1970 .

[2]  F. O. Hoffman,et al.  Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. , 1994, Risk analysis : an official publication of the Society for Risk Analysis.

[3]  H Christopher Frey,et al.  Comparison of Sensitivity Analysis Methods Based on Applications to a Food Safety Risk Assessment Model , 2004, Risk analysis : an official publication of the Society for Risk Analysis.

[4]  Joseph A. C. Delaney Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.

[5]  Scott Ferson,et al.  Sensitivity analysis for models of population viability , 1995 .

[6]  J. R. Trabalka,et al.  Methods of uncertainty analysis for a global carbon dioxide model , 1985 .

[7]  Andrea Saltelli,et al.  Sensitivity Analysis for Importance Assessment , 2002, Risk analysis : an official publication of the Society for Risk Analysis.

[8]  H Christopher Frey,et al.  Application of classification and regression trees for sensitivity analysis of the Escherichia coli O157:H7 food safety process risk model. , 2006, Journal of food protection.

[9]  Eduard Hofer,et al.  Sensitivity analysis in the context of uncertainty analysis for computationally intensive models , 1999 .

[10]  Wallace B. Whiting,et al.  Effect of uncertainties in thermodynamic data and model parameters on calculated process performance , 1993 .

[11]  Ronald L. Iman,et al.  FORTRAN 77 program and user's guide for the calculation of partial correlation and standardized regression coefficients , 1985 .

[12]  R. Forthofer,et al.  Rank Correlation Methods , 1981 .

[13]  M. Kendall,et al.  Rank Correlation Methods , 1949 .

[14]  J. L. Montagne,et al.  Emerging infectious diseases. , 1994, The Journal of infectious diseases.

[15]  L. Jaykus,et al.  The application of quantitative risk assessment to microbial food safety risks. , 1996, Critical reviews in microbiology.

[16]  C. Hedberg,et al.  Food-related illness and death in the United States. , 1999, Emerging infectious diseases.

[17]  H Christopher Frey,et al.  Introduction to Special Section on Sensitivity Analysis and Summary of NCSU/USDA Workshop on Sensitivity Analysis , 2002, Risk analysis : an official publication of the Society for Risk Analysis.

[18]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[19]  J. Durbin,et al.  Techniques for Testing the Constancy of Regression Relationships Over Time , 1975 .

[20]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[21]  A. L. Edwards,et al.  An introduction to linear regression and correlation. , 1985 .

[22]  F. J. Davis,et al.  Illustration of Sampling‐Based Methods for Uncertainty and Sensitivity Analysis , 2002, Risk analysis : an official publication of the Society for Risk Analysis.

[23]  W Schlosser,et al.  Draft risk assessment of the public health impact of Escherichia coli O157:H7 in ground beef. , 2004, Journal of food protection.

[24]  H. Müller CHANGE-POINTS IN NONPARAMETRIC REGRESSION ANALYSIS' , 1992 .

[25]  C. Loader CHANGE POINT ESTIMATION USING NONPARAMETRIC REGRESSION , 1996 .

[26]  Laura Toran,et al.  Subsurface stormflow modeling with sensitivity analysis using a Latin-hypercube sampling technique , 1996 .

[27]  Nikolay Ivanov Kolev,et al.  Uncertainty and sensitivity analysis of a postexperiment simulation of nonexplosive melt-water interaction , 1996 .

[28]  J. Neter,et al.  Applied Linear Statistical Models (3rd ed.). , 1992 .

[29]  Srikanta Mishra,et al.  Application of classification trees in the sensitivity analysis of probabilistic model results , 2003, Reliab. Eng. Syst. Saf..

[30]  John D. Graham,et al.  Going beyond the single number: Using probabilistic risk assessment to improve risk management , 1996 .

[31]  H. Frey,et al.  Characterizing, simulating, and analyzing variability and uncertainty: An illustration of methods using an air toxics emissions example , 1996 .

[32]  Michael H. Kutner Applied Linear Statistical Models , 1974 .

[33]  J. Devore,et al.  Statistics: The Exploration and Analysis of Data , 1986 .

[34]  Muni S. Srivastava,et al.  Regression Analysis: Theory, Methods, and Applications , 1991 .

[35]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[36]  S. Panchapakesan,et al.  Inference about the Change-Point in a Sequence of Random Variables: A Selection Approach , 1988 .

[37]  H. Christopher Frey,et al.  Recommended Practice Regarding Selection of Sensitivity Analysis Methods Applied to Microbial Food Safety Process Risk Models , 2005 .

[38]  Kenneth A. Rose,et al.  Parameter sensitivities, monte carlo filtering, and model forecasting under uncertainty , 1991 .

[39]  J. C. Helton,et al.  Uncertainty and sensitivity analysis in the presence of stochastic and subjective uncertainty , 1997 .

[40]  H Christopher Frey,et al.  Sensitivity Analysis of a Two‐Dimensional Probabilistic Risk Assessment Model Using Analysis of Variance , 2005, Risk analysis : an official publication of the Society for Risk Analysis.

[41]  Joan B. Rose,et al.  Quantitative Risk Assessment for Viral Contamination of Shellfish and Coastal Waters. , 1993, Journal of food protection.