A simulation model for assessing aberration detection methods used in public health surveillance for systems with limited baselines

Public health officials continue to develop and implement new types of ongoing surveillance systems in an attempt to detect aberrations in surveillance data as early as possible. In public health surveillance, aberrations are traditionally defined as an observed value being greater than an expected historical value for that same time period. To account for seasonality, traditional aberration detection methods use three or more years of baseline data across the same time period to calculate the expected historical value. Due to the recent implementation of short-term bioterrorism surveillance systems, many of the new surveillance systems have limited historical data from which to calculate an expected baseline value. Three limited baseline aberration detection methods, C1-MILD, C2-MEDIUM, and C3-ULTRA, were developed based on a one-sided positive CUSUM (cumulative sum) calculation, a commonly used quality control method used in the manufacturing industry. To evaluate the strengths and weakness of these methods, data were simulated to represent syndromic data collected through the recently developed hospital-based enhanced syndromic surveillance systems. The three methods were applied to the simulated data and estimates of sensitivity, specificity, and false-positive rates for the three methods were obtained. For the six syndromes, sensitivity for the C1-MILD, C2-MEDIUM, and C3-ULTRA models averaged 48.2, 51.3, and 53.7 per cent, respectively. Similarly, the specificities averaged 97.7, 97.8, and 96.1 per cent, respectively. The average false-positive rates for the three models were 31.8, 29.2, and 41.5 per cent, respectively. The results highlight the value and importance of developing and testing new aberration detection methods for public health surveillance data with limited baseline information.