Research Paper: A Systematic Review of the Performance Characteristics of Clinical Event Monitor Signals Used to Detect Adverse Drug Events in the Hospital Setting

OBJECTIVE Despite demonstrated benefits, few healthcare organizations have implemented clinical event monitors to detect adverse drug events (ADEs). The objective of this study was to conduct a systematic review of pharmacy and laboratory signals used by clinical event monitors to detect ADEs in hospitalized adults. DESIGN We performed a comprehensive search of MEDLINE, CINHAL and EMBASE to identify studies published between 1985 through 2006. Studies were included if they: described a clinical event monitor to detect ADEs in an adult hospital setting; described laboratory or pharmacy ADE signals; and, provided positive predictive values (PPVs) or information to allow the calculation of PPVs for individual ADE signals. MEASUREMENTS We calculated overall estimates of PPVs and 95% confidence intervals (CIs) for signals reported in 2 or more studies and contained no evidence heterogeneity. Results were examined by signal category: medication levels, laboratory tests, or antidotes. RESULTS We identified 12 observational studies describing 36 unique ADE signals. Fifteen signals (3 antidotes, 4 medication levels, and 8 laboratory values) contained no evidence of heterogeneity. The pooled PPVs for these individual signals ranged from 0.03 [CI=0.03-0.03] for hypokalemia, to 0.50 [CI=0.39-0.61] for supratherapeutic quinidine level. In general, antidotes (range=0.09-0.11) had the lowest PPVs, followed by laboratory values (0.03-0.27), and medication levels (0.03-0.50). CONCLUSION Results from this study should help clinical information system and computerized decision support producers develop or improve existing clinical event monitors to detect ADEs in their own hospitals by prioritizing those signals with the highest PPVs [corrected]

[1]  Robyn Tamblyn,et al.  Review Paper: The Impact of Electronic Health Records on Time Efficiency of Physicians and Nurses: A Systematic Review , 2005, J. Am. Medical Informatics Assoc..

[2]  M. Egger,et al.  The hazards of scoring the quality of clinical trials for meta-analysis. , 1999, JAMA.

[3]  D. Bates,et al.  Identifying drug safety issues: from research to practice. , 2000, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[4]  Asif Ahmad,et al.  Implementation of a System for Computerized Adverse Drug Event Surveillance and Intervention at an Academic Medical Center , 2006 .

[5]  D. Owens,et al.  A Normative Analytic Framework for Development of Practice Guidelines for Specific Clinical Populations , 1997, Medical decision making : an international journal of the Society for Medical Decision Making.

[6]  D. Bates,et al.  Incidence and preventability of adverse drug events among older persons in the ambulatory setting. , 2003, JAMA.

[7]  David W Bates,et al.  Linking laboratory and pharmacy: opportunities for reducing errors and improving care. , 2003, Archives of internal medicine.

[8]  G Geisslinger,et al.  Retrospective analysis of the frequency and recognition of adverse drug reactions by means of automatically recorded laboratory signals. , 1999, British journal of clinical pharmacology.

[9]  S D Small,et al.  Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. , 1995, JAMA.

[10]  Kensaku Kawamoto,et al.  Viewpoint paper: Proposal for Fulfilling Strategic Objectives of the U.S. Roadmap for National Action on Decision Support through a Service-oriented Architecture Leveraging HL7 Services , 2007, J. Am. Medical Informatics Assoc..

[11]  James W Lederer,et al.  Use of specific indicators to detect warfarin-related adverse events. , 2005, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[12]  P. Shekelle,et al.  Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care , 2006, Annals of Internal Medicine.

[13]  R S Evans,et al.  Description of a computerized adverse drug event monitor using a hospital information system. , 1992, Hospital pharmacy.

[14]  Thomas H. Payne,et al.  Development of a clinical event monitor for use with the Veterans Affairs Computerized Patient Record System and other data sources , 1998, AMIA.

[15]  Ann Aschengrau,et al.  Essentials of Epidemiology in Public Health , 2003 .

[16]  Peter M. Kilbridge,et al.  Research Paper: Automated Surveillance for Adverse Drug Events at a Community Hospital and an Academic Medical Center , 2006, J. Am. Medical Informatics Assoc..

[17]  C. Safran,et al.  Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. , 1994, Archives of internal medicine.

[18]  N. Laird,et al.  Incidence of Adverse Drug Events and Potential Adverse Drug Events: Implications for Prevention , 1995 .

[19]  George Hripcsak,et al.  Editorial Comments: Policy and the Future of Adverse Event Detection Using Information Technology , 2003, J. Am. Medical Informatics Assoc..

[20]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[21]  Stephen B. Johnson,et al.  Design of a clinical event monitor. , 1996, Computers and biomedical research, an international journal.

[22]  Michael J Becich,et al.  Pathology and patient safety: the critical role of pathology informatics in error reduction and quality initiatives. , 2004, Clinics in laboratory medicine.

[23]  J. Fleiss,et al.  The statistical basis of meta-analysis. , 1993, Statistical methods in medical research.

[24]  D. Bates,et al.  The Costs of Adverse Drug Events in Hospitalized Patients , 1997 .

[25]  P. Aspden,et al.  Preventing Medication Errors , 2007 .

[26]  D W Bates,et al.  Practical approach to determining costs and frequency of adverse drug events in a health care network. , 2001, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[27]  J Dudeck,et al.  Evaluation of clinical information systems. What can be evaluated and what cannot? , 2001, Journal of evaluation in clinical practice.

[28]  G Geisslinger,et al.  Computerized survelliance of adverse drug reactions in hospital: pilot study. , 1998, British journal of clinical pharmacology.

[29]  A R Jadad,et al.  Assessing the quality of reports of randomized clinical trials: is blinding necessary? , 1996, Controlled clinical trials.

[30]  Rainu Kaushal,et al.  Technology Evaluation: Return on Investment for a Computerized Physician Order Entry System , 2006, J. Am. Medical Informatics Assoc..

[31]  Thomas H. Payne,et al.  Review Paper: Medication-related Clinical Decision Support in Computerized Provider Order Entry Systems: A Review , 2007, J. Am. Medical Informatics Assoc..

[32]  D. Bates,et al.  Functional gaps in attaining a national health information network. , 2005, Health affairs.

[33]  David W Bates,et al.  Research Paper: Prescribers' Responses to Alerts During Medication Ordering in the Long Term Care Setting , 2006, J. Am. Medical Informatics Assoc..

[34]  H. Warner,et al.  Decision support in medicine: examples from the HELP system. , 1994, Computers and biomedical research, an international journal.

[35]  K. McDonald,et al.  Making health care safer: a critical analysis of patient safety practices. , 2001, Evidence report/technology assessment.

[36]  Chris Sauer,et al.  Research Paper: Information Technology in Complex Health Services: Organizational Impediments to Successful Technology Transfer and Diffusion , 1997, J. Am. Medical Informatics Assoc..

[37]  L. Kohn,et al.  To Err Is Human : Building a Safer Health System , 2007 .

[38]  David W. Bates,et al.  Reducing the frequency of errors in medicine using information technology. , 2001, Journal of the American Medical Informatics Association : JAMIA.

[39]  R M Gardner,et al.  Development of a computerized adverse drug event monitor. , 1991, Proceedings. Symposium on Computer Applications in Medical Care.

[40]  D. Cook,et al.  Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis. , 1995, Journal of clinical epidemiology.

[41]  Jonathan M. Teich,et al.  Identifying hospital admissions due to adverse drug events using a computer‐based monitor , 2001, AMIA.

[42]  A. Wall,et al.  Book ReviewTo Err is Human: building a safer health system Kohn L T Corrigan J M Donaldson M S Washington DC USA: Institute of Medicine/National Academy Press ISBN 0 309 06837 1 $34.95 , 2000 .

[43]  Barry L. Hainer,et al.  Primary Care Medicine: Office Evaluation and Management of the Adult Patient , 2006 .

[44]  Marc Berg,et al.  Overriding of drug safety alerts in computerized physician order entry. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[45]  David W. Bates,et al.  A computerized method for identifying incidents associated with adverse drug events in outpatients , 2001, Int. J. Medical Informatics.

[46]  Claudine McCreadie,et al.  Brocklehurst's Textbook of Geriatric medicine and gerontology , 2003 .

[47]  P. Diggle,et al.  Analysis of Longitudinal Data. , 1997 .

[48]  J. McCulloch,et al.  CHAPTER 7 – Implications for Prevention , 1972 .

[49]  D. Classen,et al.  Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. , 1997, JAMA.

[50]  R. Raschke,et al.  A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital. , 1998, JAMA.

[51]  Kay Brune,et al.  Incidence and Costs of Adverse Drug Reactions During Hospitalisation , 2000, Drug safety.

[52]  George Hripcsak,et al.  Review Paper: Detecting Adverse Events Using Information Technology , 2003, J. Am. Medical Informatics Assoc..

[53]  Mathews Ka,et al.  Evaluation of clinical information systems. , 1993 .

[54]  Linda Gavendo,et al.  The incidence of adverse drug events in two large academic long-term care facilities. , 2005, The American journal of medicine.

[55]  J. Hanlon,et al.  Epidemiology of medication-related adverse events in nursing homes. , 2006, The American journal of geriatric pharmacotherapy.

[56]  R S Evans,et al.  Prevention of adverse drug events through computerized surveillance. , 1992, Proceedings. Symposium on Computer Applications in Medical Care.

[57]  D. Bates,et al.  Improving response to critical laboratory results with automation: results of a randomized controlled trial. , 1999, Journal of the American Medical Informatics Association : JAMIA.

[58]  Steven H. Brown,et al.  RADARx: Recognizing, Assessing, and Documenting Adverse Rx events , 2000, AMIA.

[59]  Petra A. Thuermann,et al.  Detection of Adverse Drug Reactions in a Neurological Department , 2002, Drug safety.

[60]  P. Corey,et al.  Incidence of Adverse Drug Reactions in Hospitalized Patients , 2012 .

[61]  S. Zeger,et al.  Longitudinal data analysis using generalized linear models , 1986 .

[62]  G. Geisslinger,et al.  Computerized surveillance of adverse drug reactions in hospital: Implementation , 1999, European Journal of Clinical Pharmacology.

[63]  E. Hahn,et al.  Implementation of a computer‐assisted monitoring system for the detection of adverse drug reactions in gastroenterology , 2004, Alimentary pharmacology & therapeutics.

[64]  Jonathan M. Teich,et al.  Research Paper: Identifying Adverse Drug Events: Development of a Computer-based Monitor and Comparison with Chart Review and Stimulated Voluntary Report , 1998, J. Am. Medical Informatics Assoc..

[65]  A. Egberts,et al.  Adverse drug events in hospitalized patients A comparison of doctors, nurses and patients as sources of reports , 1999, European Journal of Clinical Pharmacology.

[66]  S D Small,et al.  The incident reporting system does not detect adverse drug events: a problem for quality improvement. , 1995, The Joint Commission journal on quality improvement.

[67]  P Glasziou,et al.  Meta-analytic methods for diagnostic test accuracy. , 1995, Journal of clinical epidemiology.

[68]  Peter J. Haug,et al.  Decision support in medicine: lessons from the HELP system , 2003, Int. J. Medical Informatics.

[69]  Peter Jüni,et al.  Quality of reporting of randomized trials as a measure of methodologic quality. , 2002, JAMA.

[70]  Alastair Baker,et al.  Crossing the Quality Chasm: A New Health System for the 21st Century , 2001, BMJ : British Medical Journal.

[71]  Alex J Sutton,et al.  Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. , 2002, International journal of epidemiology.

[72]  Paul N. Gorman,et al.  Computerized physician order entry in U.S. hospitals: results of a 2002 survey. , 2003, Journal of the American Medical Informatics Association : JAMIA.

[73]  John F Hurdle,et al.  High rates of adverse drug events in a highly computerized hospital. , 2005, Archives of internal medicine.

[74]  S. Lipsitz,et al.  Performance of generalized estimating equations in practical situations. , 1994, Biometrics.

[75]  Jalpa A Doshi,et al.  National Estimates of Medication Use in Nursing Homes: Findings from the 1997 Medicare Current Beneficiary Survey and the 1996 Medical Expenditure Survey , 2005, Journal of the American Geriatrics Society.

[76]  Adam Wright,et al.  White paper: A Roadmap for National Action on Clinical Decision Support , 2007, J. Am. Medical Informatics Assoc..

[77]  N. Laird,et al.  Incidence of adverse drug events and potential adverse drug events , 1995 .

[78]  Stanley M. Huff,et al.  White Paper: Toward Vocabulary Domain Specifications for Health Level 7 - coded Data Elements , 2000, J. Am. Medical Informatics Assoc..

[79]  W. Churchill,et al.  Computer-based system for preventing adverse drug events. , 2004, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[80]  E Pfeiffer,et al.  Pharmacology of aging. , 1980, Frontiers in aging series.

[81]  Frederick Mosteller,et al.  Guidelines for Meta-analyses Evaluating Diagnostic Tests , 1994, Annals of Internal Medicine.

[82]  Michael Pignone,et al.  Screening for Colorectal Cancer in Adults at Average Risk: A Summary of the Evidence for the U.S. Preventive Services Task Force , 2002, Annals of Internal Medicine.

[83]  David W. Bates,et al.  Research Paper: Using Computerized Data to Identify Adverse Drug Events in Outpatients , 2001, J. Am. Medical Informatics Assoc..