Quantifying and Reporting Uncertainty from Systematic Errors

Optimal use of epidemiologic findings in decision making requires more information than standard analyses provide. It requires calculating and reporting the total uncertainty in the results, which in turn requires methods for quantifying the uncertainty introduced by systematic error. Quantified uncertainty can improve policy and clinical decisions, better direct further research, and aid public understanding, and thus enhance the contributions of epidemiology. The error quantification approach proposed here is based on estimating a probability distribution for a bias-corrected effect measure based on externally-derived distributions of bias levels. Using Monte Carlo simulation, corrections for multiple biases are combined by identifying the steps through which true causal effects become data, and (in reverse order) correcting for the errors introduced by each step. The bias-correction calculations are the same as those used in sensitivity analysis, but the resulting distribution of possible true values is more than a sensitivity analysis; it is a more complete reporting of the actual study results. The approach is illustrated with an application to a recent study that resulted in the drug, phenylpropanolamine, being removed from the market.

[1]  Carl V Phillips,et al.  Epidemiologic Perspectives & Innovations Open Access the Missed Lessons of Sir Austin Bradford Hill , 2004 .

[2]  C V Phillips,et al.  The economics of 'more research is needed'. , 2001, International journal of epidemiology.

[3]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[4]  T Brott,et al.  Phenylpropanolamine and the risk of hemorrhagic stroke. , 2000, The New England journal of medicine.

[5]  Sander Greenland,et al.  Response: Defining and estimating causal effects , 2002 .

[6]  Ricardo Rodrı́guez Iglesias,et al.  Should we risk it? Exploring environmental, health, and technological problem solving , 2001 .

[7]  D. Vose Risk Analysis: A Quantitative Guide , 2000 .

[8]  George Maldonado,et al.  Occupational exposure to glycol ethers and human congenital malformations , 2003, International archives of occupational and environmental health.

[9]  Timothy L Lash,et al.  Semi-Automated Sensitivity Analysis to Assess Systematic Errors in Observational Data , 2003, Epidemiology.

[10]  T. Lash,et al.  A Sensitivity Analysis to Separate Bias Due to Confounding from Bias Due to Predicting Misclassification by a Variable That Does Both , 2000, Epidemiology.

[11]  S Greenland,et al.  Sensitivity Analysis, Monte Carlo Risk Analysis, and Bayesian Uncertainty Assessment , 2001, Risk analysis : an official publication of the Society for Risk Analysis.

[12]  S. Sidney,et al.  Incidence of stroke and myocardial infarction in women of reproductive age. , 1997, Stroke.

[13]  R. Zeckhauser,et al.  A primer for policy analysis , 1978 .

[14]  Sander Greenland,et al.  The Impact of Prior Distributions for Uncontrolled Confounding and Response Bias , 2003 .

[15]  Carl V Phillips,et al.  Quantifying errors without random sampling , 2003, BMC medical research methodology.

[16]  B. Fischhoff,et al.  Rating the Risks , 1979 .

[17]  J. Riley,et al.  The analytics of uncertainty and information: Long-run relationships and the credibility of threats and promises , 1992 .

[18]  S. Greenland,et al.  Estimating causal effects. , 2002, International journal of epidemiology.

[19]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[20]  P. Slovic Perception of risk. , 1987, Science.

[21]  S. Schneeweiss,et al.  Causation of Bias: The Episcope , 2001, Epidemiology.