Evaluating Detection and Diagnostic Decision Support Systems for Bioterrorism Response

We evaluated the usefulness of detection systems and diagnostic decision support systems for bioterrorism response. We performed a systematic review by searching relevant databases (e.g., MEDLINE) and Web sites for reports of detection systems and diagnostic decision support systems that could be used during bioterrorism responses. We reviewed over 24,000 citations and identified 55 detection systems and 23 diagnostic decision support systems. Only 35 systems have been evaluated: 4 reported both sensitivity and specificity, 13 were compared to a reference standard, and 31 were evaluated for their timeliness. Most evaluations of detection systems and some evaluations of diagnostic systems for bioterrorism responses are critically deficient. Because false-positive and false-negative rates are unknown for most systems, decision making on the basis of these systems is seriously compromised. We describe a framework for the design of future evaluations of such systems.

[1]  A S Elstein,et al.  The influence of a decision support system on the differential diagnosis of medical practitioners at three levels of training. , 1996, Proceedings : a conference of the American Medical Informatics Association. AMIA Fall Symposium.

[2]  W. DuMouchel,et al.  Unlocking Clinical Data from Narrative Reports: A Study of Natural Language Processing , 1995, Annals of Internal Medicine.

[3]  J. Taylor,et al.  Knowledge-based patient screening for rare and emerging infectious/parasitic diseases: a case study of brucellosis and murine typhus. , 1997, Emerging infectious diseases.

[4]  D. S. Shapiro,et al.  Evaluation of the computer program GIDEON (Global Infectious Disease and Epidemiology Network) for the diagnosis of fever in patients admitted to a medical service. , 1998, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[5]  R. E. Ashton,et al.  Initial use of a computer system for assisting dermatological diagnosis in general practice , 2000, Medical informatics and the Internet in medicine.

[6]  K Nakamura,et al.  Effect of an artificial neural network on radiologists' performance in the differential diagnosis of interstitial lung disease using chest radiographs. , 1999, AJR. American journal of roentgenology.

[7]  Jeffrey R. Hammersley,et al.  Evaluating the Utility of Available Differential Diagnosis Systems. , 1988 .

[8]  Evaluation of the Computer Program GIDEON for the Diagnosis of Fever in Patients Admitted to a Medical Service , 1998 .

[9]  Les M. Sztandera,et al.  Using fuzzy sets to analyze putative correlates between age, blood type, gender and/or race with bacterial infection , 2001, Artif. Intell. Medicine.

[10]  J. R. Scotti,et al.  Available From , 1973 .

[11]  W. Kapoor,et al.  A computer-assisted medical diagnostic consultation service. Implementation and prospective evaluation of a prototype. , 1989, Annals of internal medicine.

[12]  Omar Bouhaddou,et al.  Consumer health informatics: knowledge engineering and evaluation studies of medical HouseCall , 1998, AMIA.

[13]  J. Valdés,et al.  Agroterrorism: Agricultural Infrastructure Vulnerability , 1999, Annals of the New York Academy of Sciences.

[14]  F. Sanz,et al.  Validation of the medical expert system PNEUMON-IA. , 1992, Computers and biomedical research, an international journal.

[15]  F. Ligler,et al.  An array immunosensor for simultaneous detection of clinical analytes. , 1999, Analytical chemistry.

[16]  J. Gerberding,et al.  Anthrax Bioterrorism: Lessons Learned and Future Directions , 2002, Emerging infectious diseases.

[17]  S Katsuragawa,et al.  Computer-aided diagnosis for detection of interstitial opacities on chest radiographs. , 1998, AJR. American journal of roentgenology.

[18]  Livermore ’ s strategy for defense against the use of biological weapons integrates technology , operations , and policy and provides a framework for coordinated local , state , and federal emergency response . Reducing the Threat of Biological Weapons , 2022 .

[19]  Tanja Popovic,et al.  Public Health in the Time of Bioterrorism , 2002, Emerging infectious diseases.

[20]  Peter J. Haug,et al.  Automatic identification of pneumonia related concepts on chest x-ray reports , 1999, AMIA.

[21]  R. Pethybridge,et al.  DERMIS: a computer system for assisting primary‐care physicians with dermatological diagnosis , 1992, The British journal of dermatology.

[22]  Lennart Ljung,et al.  Bacteria classification based on feature extraction from sensor data , 1998 .

[23]  A. L. Baker,et al.  Performance of four computer-based diagnostic systems. , 1994, The New England journal of medicine.

[24]  Debra M. Niemeyer,et al.  Laboratory Response to Anthrax Bioterrorism, New York City, 2001 , 2002, Emerging infectious diseases.

[25]  M. Musen,et al.  Bioterrorism preparedness and response: use of information technologies and decision support systems. , 2002, Evidence report/technology assessment.

[26]  A. El‐Solh,et al.  Predicting active pulmonary tuberculosis using an artificial neural network. , 1999, Chest.

[27]  N L Jain,et al.  Respiratory Isolation of Tuberculosis Patients Using Clinical Guidelines and an Automated Clinical Decision Support System , 1998, Infection Control & Hospital Epidemiology.

[28]  G. Chapman,et al.  [Medical decision making]. , 1976, Lakartidningen.

[29]  Peter J. Haug,et al.  Comparing expert systems for identifying chest x-ray reports that support pneumonia , 1999, AMIA.

[30]  Berger,et al.  Computer Program for Diagnosing and Teaching Geographic Medicine. , 1995, Journal of travel medicine.