How does improved forecasting benefit detection? An application to biosurveillance

While many methods have been proposed for detecting disease outbreaks from pre-diagnostic data, their performance is usually not well understood. We argue that most existing temporal detection methods for biosurveillance can be characterized as a forecasting component coupled with a monitoring/detection component. In this paper, we describe the effect of forecast accuracy on detection performance. Quantifying this effect allows one to measure the benefits of improved forecasting and determine when it is worth improving a forecast method's precision at the cost of robustness or simplicity. We quantify the effect of forecast accuracy on detection metrics under different scenarios and investigate the effect when standard assumptions are violated. We illustrate our results by examining performance on authentic biosurveillance data.

[1]  W. J. Padgett,et al.  On the α-risks for shewhart control charts , 1992 .

[2]  Galit Shmueli,et al.  Statistical issues and challenges associated with rapid detection of bio‐terrorist attacks , 2005, Statistics in medicine.

[3]  Joseph S Lombardo,et al.  ESSENCE II and the framework for evaluating syndromic surveillance systems. , 2004, MMWR supplements.

[4]  Rassoul Noorossana,et al.  Effect of Autocorrelation on Performance of the MCUSUM Control Chart , 2006, Qual. Reliab. Eng. Int..

[5]  Galit Shmueli,et al.  Automated time series forecasting for biosurveillance , 2007, Statistics in medicine.

[6]  Sean Murphy,et al.  Preparing Biosurveillance Data for Classic Monitoring , 2007 .

[7]  J C Benneyan,et al.  Statistical Quality Control Methods in Infection Control and Hospital Epidemiology, Part II: Chart Use, Statistical Properties, and Research Issues , 1998, Infection Control & Hospital Epidemiology.

[8]  Ronald D Fricker,et al.  Comparing syndromic surveillance detection methods: EARS' versus a CUSUM‐based methodology , 2008, Statistics in medicine.

[9]  Alyson G. Wilson,et al.  Statistical methods in counterterrorism : game theory, modeling, syndromic surveillance, and biometric authentication , 2006 .

[10]  D. Siegmund.  Sequential Analysis: Tests and Confidence Intervals , 1985 .

[11]  Robert H. Smith.  Statistical Challenges in Modern Biosurveillance Galit Shmueli Department of Decision & Information Technologies and The Center for Health Information and Decision Systems , 2007 .

[12]  Douglas C. Montgomery,et al.  Introduction to Statistical Quality Control , 1986 .

[13]  Wei Jiang,et al.  A One-Sided EWMA Control Chart for Monitoring Process Means , 2007, Commun. Stat. Simul. Comput..

[14]  Ronald D. Fricker,et al.  Evaluating Statistical Methods for Syndromic Surveillance , 2006 .

[15]  George E. P. Box,et al.  Statistical Control: By Monitoring and Feedback Adjustment , 1997 .

[16]  Galit Shmueli,et al.  Early statistical detection of anthrax outbreaks by tracking over-the-counter medication sales , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Douglas C. Montgomery,et al.  Some Statistical Process Control Methods for Autocorrelated Data , 1991 .

[18]  Kenneth D. Mandl,et al.  Time series modeling for syndromic surveillance , 2003, BMC Medical Informatics Decis. Mak..

[19]  Roger M. Sauter,et al.  Introduction to Statistical Quality Control (2nd ed.) , 1992 .

[20]  Loon Ching Tang,et al.  Statistical process control and forecasting , 1997 .

[21]  S. Crowder.  A simple method for studying run-length distribution of exponentially weighted moving average charts , 1987 .

[22]  Galit Shmueli,et al.  Ensemble Forecasting for Disease Outbreak Detection , 2008, AAAI.

[23]  William H. Woodall,et al.  The Use of Control Charts in Health-Care and Public-Health Surveillance , 2006 .

[24]  William H. Woodall,et al.  The effect of autocorrelation on the retrospective X-chart , 1992 .

[25]  C. Chatfield,et al.  Prediction intervals for multiplicative Holt-Winters , 1991 .

[26]  S. Crowder.  Average Run Lengths of Exponentially Weighted Moving Average Control Charts , 1987 .

[27]  L. Hutwagner,et al.  The bioterrorism preparedness and response Early Aberration Reporting System (EARS) , 2003, Journal of Urban Health.

[28]  J. Pavlin,et al.  Bio-ALIRT biosurveillance detection algorithm evaluation. , 2004, MMWR supplements.

[29]  C. S. van Dobben de Bruyn.  The Interplay of Tracking Signals and Adaptive Predictors , 1967 .