Evaluation of rule effectiveness and positive predictive value of clinical rules in a Dutch clinical decision support system in daily hospital pharmacy practice

INTRODUCTION Our advanced clinical decision support (CDS) system, entitled 'adverse drug event alerting system' (ADEAS), is in daily use in our hospital pharmacy. It is used by hospital pharmacists to select patients at risk of possible adverse drug events (ADEs). The system retrieves data from several information systems, and uses clinical rules to select the patients at risk of ADEs. The clinical rules are all medication related and are formulated using seven risk categories. OBJECTIVE This studies objectives are to 1) evaluate the use of the CDS system ADEAS in daily hospital pharmacy practice, and 2) assess the rule effectiveness and positive predictive value (PPV) of the clinical rules incorporated in the system. SETTING Leiden University Medical Center, The Netherlands. All patients admitted on six different internal medicine and cardiology wards were included. MEASURES Outcome measures were total number of alerts, number of patients with alerts and the outcome of these alerts: whether the hospital pharmacist gave advice to prevent a possible ADE or not. Both overall rule effectiveness and PPV and rule effectiveness and PPV per clinical rule risk category were scored. STUDY DESIGN During a 5 month study period safety alerts were generated daily by means of ADEAS. All alerts were evaluated by a hospital pharmacist and if necessary, healthcare professionals were subsequently contacted and advice was given in order to prevent possible ADEs. RESULTS During the study period ADEAS generated 2650 safety alerts in 931 patients. In 270 alerts (10%) the hospital pharmacist contacted the physician or nurse and in 204 (76%) cases this led to an advice to prevent a possible ADE. The remaining 2380 alerts (90%) were scored as non-relevant. Most alerts were generated with clinical rules linking pharmacy and laboratory data (1685 alerts). The overall rule effectiveness was 0.10 and the overall PPV was 0.08. Combination of rule effectiveness and PPV was highest for clinical rules based upon the risk category "basic computerized physician order entry (CPOE) medication safety alerts fine-tuned to high risk patients" (rule efficiency=0.17; PPV=0.14). CONCLUSION ADEAS can effectively be used in daily hospital pharmacy practice to select patients at risk of potential ADEs, but to increase the benefits for routine patient care and to increase efficiency, both rule effectiveness and PPV for the clinical rules should be improved. Furthermore, clinical rules would have to be refined and restricted to those categories that are potentially most promising for clinical relevance, i.e. "clinical rules with a combination of pharmacy and laboratory data" and "clinical rules based upon the basic CPOE medication safety alerts fine-tuned to high risk patients".

[1]  Diane L. Seger,et al.  Application of Information Technology: Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care , 2006, J. Am. Medical Informatics Assoc..

[2]  J. Brouwers,et al.  Clinical Relevance of Drug-Drug Interactions , 2005, Drug safety.

[3]  H. Guchelaar,et al.  The potential role of computerisation and information technology in improving prescribing in hospitals , 2003, Pharmacy World and Science.

[4]  Arie Hasman,et al.  Design and implementation of a framework to support the development of clinical guidelines , 2001, Int. J. Medical Informatics.

[5]  Mor Peleg,et al.  A pattern-based analysis of clinical computer-interpretable guideline modelling languages , 2006 .

[6]  M. D. Kalmeijer,et al.  Implementation of a computerized physician medication order entry system at the Academic Medical Centre in Amsterdam , 2004, Pharmacy World and Science.

[7]  P. Mol,et al.  Comparison of methods for identifying patients at risk of medication-related harm , 2010, Quality and Safety in Health Care.

[8]  W. Churchill,et al.  Computer-based system for preventing adverse drug events. , 2004, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[9]  A Hasman,et al.  GASTON: an architecture for the acquisition and execution of clinical guideline-application tasks. , 2000, Medical informatics and the Internet in medicine.

[10]  Wil M. P. van der Aalst,et al.  Research Paper: A Pattern-based Analysis of Clinical Computer-interpretable Guideline Modeling Languages , 2007, J. Am. Medical Informatics Assoc..

[11]  V. Colucci,et al.  Using serum creatinine concentrations to screen for inappropriate dosage of renally eliminated drugs. , 1991, American journal of hospital pharmacy.

[12]  Monitoring abnormal laboratory values as antecedents to drug-induced injury. , 2005, The Journal of trauma.

[13]  P A De Clercq,et al.  Development of a computerised alert system, ADEAS, to identify patients at risk for an adverse drug event , 2010, Quality and Safety in Health Care.

[14]  Jens Kaltschmidt,et al.  Successful strategy to improve the specificity of electronic statin–drug interaction alerts , 2009, European Journal of Clinical Pharmacology.

[15]  Marc Berg,et al.  Drug safety alert generation and overriding in a large Dutch university medical centre , 2009, Pharmacoepidemiology and drug safety.

[16]  D. Bates,et al.  Prioritizing strategies for preventing medication errors and adverse drug events in pediatric inpatients. , 2003, Pediatrics.

[17]  C. M. Cheng.  Hospital Systems for the Detection and Prevention of Adverse Drug Events , 2011, Clinical pharmacology and therapeutics.

[18]  David W. Bates,et al.  Can surveillance systems identify and avert adverse drug events? A prospective evaluation of a commercial application. , 2008, Journal of the American Medical Informatics Association : JAMIA.

[19]  Wei Wu,et al.  The Effect of Computerized Physician Order Entry with Clinical Decision Support on the Rates of Adverse Drug Events: A Systematic Review , 2008, Journal of General Internal Medicine.

[20]  Michael T. Johnson,et al.  Computer-based program for identifying medication orders requiring dosage modification based on renal function. , 1991, American journal of hospital pharmacy.

[21]  Thomas H. Payne,et al.  Review Paper: Medication-related Clinical Decision Support in Computerized Provider Order Entry Systems: A Review , 2007, J. Am. Medical Informatics Assoc..

[22]  T. Egberts,et al.  Drug-Related Problems in Hospitalised Patients , 2000, Drug safety.

[23]  Martin Jung,et al.  RESEARCH ARTICLE Open Access Development of a context model to prioritize drug safety alerts in CPOE systems , 2022 .

[24]  ELSKE AMMENWERTH,et al.  Review Paper: The Effect of Electronic Prescribing on Medication Errors and Adverse Drug Events: A Systematic Review , 2008, J. Am. Medical Informatics Assoc..

[25]  Asif Ahmad,et al.  Implementation of a System for Computerized Adverse Drug Event Surveillance and Intervention at an Academic Medical Center , 2006 .

[26]  P. Hudson,et al.  Medication errors: hospital pharmacist perspective. , 2005, Drugs.

[27]  G D Schiff,et al.  Prescribing potassium despite hyperkalemia: medication errors uncovered by linking laboratory and pharmacy information systems. , 2000, The American journal of medicine.

[28]  D. Bates,et al.  Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. , 2003, Archives of internal medicine.

[29]  David W Bates,et al.  Linking laboratory and pharmacy: opportunities for reducing errors and improving care. , 2003, Archives of internal medicine.

[30]  Marc Berg,et al.  Overriding of drug safety alerts in computerized physician order entry. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[31]  Henk-Jan Guchelaar,et al.  A Computerized Adverse Drug Event Alerting System Using Clinical Rules , 2011, Drug safety.