Systematic Review: Surveillance Systems for Early Detection of Bioterrorism-Related Diseases

Key Summary Points The practice of surveillance is changing to address the threat of bioterrorism and to take advantage of the increasing availability of electronic data. The authors identified published descriptions of 29 systems designed specifically for bioterrorism surveillance. Bioterrorism surveillance systems either monitor the incidence of bioterrorism-related syndromes (9) or monitor environmental samples for bioterrorism agents (20). Only 2 syndromic surveillance systems and no environmental monitoring system were evaluated in peer-reviewed studies. Both evaluations of syndromic surveillance systems compared the incidence of flu-like illness syndromes with results from national influenza surveillance. Existing evaluations of surveillance systems for detecting bioterrorism are insufficient to characterize the performance of these systems. Evaluation of bioterrorism surveillance is needed to inform decisions about deploying systems and to facilitate decision making on the basis of system results. The anthrax attacks of 2001 and the recent outbreaks of severe acute respiratory syndrome (SARS) and influenza strikingly demonstrate the continuing threat from illnesses resulting from bioterrorism and related infectious diseases. In particular, these outbreaks have highlighted that an essential component of preparations for illnesses and syndromes potentially related to bioterrorism includes the deployment of surveillance systems that can rapidly detect and monitor the course of an outbreak and thus minimize associated morbidity and mortality (1-3). Driven by the threat of additional outbreaks resulting from bioterrorism and the increasing availability of data available for surveillance, surveillance systems have proliferated. The Centers for Disease Control and Prevention (CDC) defines surveillance systems as those that collect and analyze morbidity, mortality, and other relevant data and facilitate the timely dissemination of results to appropriate decision makers (3, 4). However, there is little consensus as to which sources of surveillance data or which collection, analysis, and reporting technologies are probably the most timely, sensitive, and specific for detecting and managing bioterrorism-related illness and related emerging infectious diseases (5). Existing surveillance systems for bioterrorism-related diseases vary widely with respect to the methods used to collect the surveillance data, surveillance characteristics of the data collected, and analytic methods used to determine when a potential outbreak has occurred. Traditionally, the primary method for collecting surveillance data was manual reporting of suspicious and notifiable clinical and laboratory data from clinicians, hospitals, and laboratories to public health officials (6). Recent innovations in disease surveillance that may improve the timeliness, sensitivity, and specificity of bioterrorism-related outbreak detection include surveillance for syndromes rather than specific diseases and the automated extraction and analysis of routinely collected clinical, administrative, pharmacy, and laboratory data. Little is known about the accuracy of surveillance systems for bioterrorism and related emerging infectious diseases, perhaps because of the diversity of potential data sources for bioterrorism surveillance data; methods for their analysis; and the uncertainty about the costs, benefits, and detection characteristics of each. Under the auspices of the University of California, San FranciscoStanford Evidence-based Practice Center, we prepared a comprehensive systematic review that evaluated the ability of available information technologies to inform clinicians and public health officials who are preparing for and responding to bioterrorism and related emerging infectious diseases (7). In this paper, we present the available data on existing systems for surveillance of illnesses and syndromes potentially related to bioterrorism and the published evaluation data on these systems. Methods We sought to identify published reports of surveillance systems designed to collect, analyze, and report surveillance data for bioterrorism-related diseases or syndromes or reports of surveillance systems for naturally occurring diseases, if potentially useful for bioterrorism surveillance. We used the U.S. Department of Health and Human Services' definition of bioterrorism-related diseases (8-10). Because most patients with bioterrorism-related diseases initially present with influenza-like illness, acute respiratory distress, gastrointestinal symptoms, febrile hemorrhagic syndromes, and febrile illnesses with either dermatologic or neurologic findings, we considered these conditions to be the bioterrorism-related syndromes. We briefly summarize our methods, which are described in detail elsewhere (7). Literature Sources and Search Strategies We searched 3 sources for relevant reports: 5 databases of peer-reviewed articles (for example, MEDLINE, GrayLIT, and National Technical Information Service), government reports, and Web sites of relevant government and commercial entities. We consulted public health, bioterrorism preparedness, and national security experts to identify the 16 government agencies most likely to fund, develop, or use bioterrorism systems (for example, CDC and U.S. Department of Defense). We searched the Web sites of these government agencies and other academic and commercial sites. Finally, we identified additional articles from the bibliographies of included articles and from conference proceedings. We developed 2 separate search strategies: 1 for MEDLINE (January 1985 to April 2002) and 1 for other sources. In both searches, we included terms such as bioterrorism, biological warfare, information technology, surveillance, public health, and epidemiology. Complete search strategies are available from the authors (7). Study Selection and Data Abstraction We reviewed titles, abstracts, and full-length articles to identify potentially relevant articles. Two abstractors, who were blinded to the study authors, abstracted data from all included peer-reviewed articles onto pretested abstraction forms. Given the large volume of Web sites screened, only 1 abstractor, whose work was frequently reviewed by a colleague, collected data from each Web-based report. Evaluation of Reports of Surveillance Systems The CDC developed a draft guideline for evaluating public health surveillance systems (3, 11, 12). This guideline recommends that reports of surveillance systems include the following: descriptions of the public health importance of the health event under surveillance; the system under evaluation; the direct costs needed to operate the system; the usefulness of the system; and evaluations of the system's simplicity, flexibility (that is, the system's ability to change as surveillance needs change), acceptability (as reflected by the willingness of participants and stakeholders to contribute to the data collection, analysis and use), sensitivity to detect outbreaks, positive predictive value of system alarms for true outbreaks, representativeness of the population covered by the system, and timeliness of detection (11, 12). The guideline describes these key elements to consider in an evaluation of a surveillance system but does not provide specific scoring or an evaluation tool. We abstracted information about each CDC criterion from each included reference. Data Synthesis We reviewed 17510 citations of peer-reviewed articles and 8088 Web sites, of which 192 reports on 115 surveillance systems met our inclusion criteria (Figure 1). Of these, 29 systems were designed specifically for detecting bioterrorism-related diseases (as defined by the U.S. Department of Health and Human Services [8-10]) or bioterrorism-related syndromes (for example, flu-like syndrome and fever with rash). An additional 86 systems were designed for surveillance of naturally occurring illnesses, but elements of their design, deployment, or evaluations may be relevant for implementing or evaluating bioterrorism surveillance systems. For example, we included reports of systems for surveillance of nonbiothreat pathogens if they were designed to rapidly transmit surveillance data from sources that could be useful for detecting bioterrorism-related illness (for example, laboratory data, clinicians' reports, hospital-based data, or veterinary data) or if they reported methods of spatial or temporal analyses that facilitated rapid and accurate decision making by public health users. We present the evidence about the systems designed principally for bioterrorism surveillance systems and summarize the evidence about the other surveillance systems. Figure 1. Search results. Surveillance Systems Designed for Bioterrorism-Related Diseases or Syndromes We identified 2 types of systems for surveillance of bioterrorism-related diseases or syndromes: those that monitor the incidence of bioterrorism-related syndromes and those that collect and transmit bioterrorism detection data from environmental or clinical samples to decision makers. Surveillance Systems Collecting Syndromic Reports The 9 surveillance systems designed to monitor the incidence of bioterrorism-related syndromes vary widely with respect to syndromes under surveillance, data collected, flexibility of the data collection tool (for example, some Web-based systems allow remote users to change the prompts given to data collectors), acceptability to data collectors, and methods used to analyze the data (13-23) (Table). Table. Surveillance Systems Collecting Syndromic Reports* Two syndromic surveillance systems were evaluated in peer-reviewed reports: the National Health Service Direct system and the program of systematic surveillance of International Classification of Diseases, Ninth Revision (ICD-9), codes from the electronic medical records of the Harvard Vanguard Medical Associates (20, 23). In these evaluations, the numbers of