Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review

Objective Clinical decision support (CDS) hard-stop alerts-those in which the user is either prevented from taking an action altogether or allowed to proceed only with the external override of a third party-are increasingly common but can be problematic. To understand their appropriate application, we asked 3 key questions: (1) To what extent are hard-stop alerts effective in improving patient health and healthcare delivery outcomes? (2) What are the adverse events and unintended consequences of hard-stop alerts? (3) How do hard-stop alerts compare to soft-stop alerts? Methods and Materials Studies evaluating computerized hard-stop alerts in healthcare settings were identified from biomedical and computer science databases, gray literature sites, reference lists, and reviews. Articles were extracted for process outcomes, health outcomes, unintended consequences, user experience, and technical details. Results Of 32 studies, 15 evaluated health outcomes, 16 process outcomes only, 10 user experience, and 4 compared hard and soft stops. Seventy-nine percent showed improvement in health outcomes and 88% in process outcomes. Studies reporting good user experience cited heavy user involvement and iterative design. Eleven studies reported on unintended consequences including avoidance of hard-stopped workflow, increased alert frequency, and delay to care. Hard stops were superior to soft stops in 3 of 4 studies. Conclusions Hard stops can be effective and powerful tools in the CDS armamentarium, but they must be implemented judiciously with continuous user feedback informing rapid, iterative design. Investigators must report on associated health outcomes and unintended consequences when implementing IT solutions to clinical problems.

[1]  Jesús Favela,et al.  Using Mixed Methods in Health Information Technology Evaluation , 2016, Nursing Informatics.

[2]  J. Meredith,et al.  Surgical Intensive Care Unit Mobility is Increased after Institution of a Computerized Mobility Order Set and Intensive Care Unit Mobility Protocol: A Prospective Cohort Analysis , 2010, The American surgeon.

[3]  Jeffrey B. Weilburg,et al.  Increasing the appropriateness of outpatient imaging: effects of a barrier to ordering low-yield examinations. , 2010, Radiology.

[4]  Mary K. Lovejoy,et al.  Implementation of a Mandated Venous Thromboembolism Clinical Order Set Improves Venous Thromboembolism Core Measures , 2014, Hospital practice.

[5]  Charles E. Leonard,et al.  Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. , 2010, Archives of internal medicine.

[6]  A W Kushniruk,et al.  Methods for Addressing Technology-induced Errors: The Current State , 2016, Yearbook of Medical Informatics.

[7]  George Hripcsak,et al.  The Role of Housestaff in Implementing Medication Reconciliation on Admission at an Academic Medical Center , 2011, American journal of medical quality : the official journal of the American College of Medical Quality.

[8]  M. Silverman,et al.  Using clinical decision support as a means of implementing a universal postpartum depression screening program , 2016, Archives of Women's Mental Health.

[9]  Mark A Tully,et al.  Failure of a numerical quality assessment scale to identify potential risk of bias in a systematic review: a comparison study , 2015, BMC Research Notes.

[10]  J. Marriott,et al.  The effectiveness of computerised decision support on antibiotic use in hospitals: A systematic review , 2017, PloS one.

[11]  Paul A Thompson,et al.  Decreasing the critical value of hemoglobin required for physician notification reduces the rate of blood transfusions , 2016, International journal of general medicine.

[12]  Daniel Kurnik,et al.  Prescriber response to computerized drug alerts for electronic prescriptions among hospitalized patients , 2017, Int. J. Medical Informatics.

[13]  Michelle Sweidan,et al.  Drug interaction alerts in software — what do general practitioners and pharmacists want? , 2011, The Medical journal of Australia.

[14]  Brad Stevinson,et al.  Does a Clinical Decision Rule Using D-Dimer Level Improve the Yield of Pulmonary CT Angiography? , 2011 .

[15]  J. Devlin,et al.  Use of Computer Alerts to Prevent the Inappropriate Use of Metformin in an Inpatient Setting , 2012, Quality management in health care.

[16]  Christoph U. Lehmann,et al.  Eliminating Health Care Disparities With Mandatory Clinical Decision Support: The Venous Thromboembolism (VTE) Example , 2015, Medical care.

[17]  Shane P Stenner,et al.  ePrescribing: Reducing Costs through In-Class Therapeutic Interchange , 2016, Applied Clinical Informatics.

[18]  Peter J Pronovost,et al.  Improved prophylaxis and decreased rates of preventable harm with the use of a mandatory computerized clinical decision support tool for prophylaxis for venous thromboembolism in trauma. , 2012, Archives of surgery.

[19]  Elizabeth M. Swisher,et al.  Reflex test reminders in required cancer synoptic templates decrease order entry error: An analysis of mismatch repair immunohistochemical orders to screen for Lynch syndrome , 2016, Journal of pathology informatics.

[20]  Issam Makhoul,et al.  Improving Communication on Intent of Chemotherapy Using QOPI Scores and PDSA Cycles , 2016, Journal of Cancer Education.

[21]  C. Gibson,et al.  Appropriate utilisation of cardiac telemetry monitoring: a quality improvement project , 2019, BMJ open quality.

[22]  Neil M. Paige,et al.  Electronic health record-based interventions for improving appropriate diagnostic imaging: a systematic review and meta-analysis. , 2015, Annals of internal medicine.

[23]  Safe use of health information technology. , 2015, Sentinel event alert.

[24]  Edith Nutescu,et al.  Effects of clinical decision support on venous thromboembolism risk assessment, prophylaxis, and prevention at a university teaching hospital. , 2010, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[25]  Gary W Procop,et al.  Reducing duplicate testing: a comparison of two clinical decision support tools. , 2015, American journal of clinical pathology.

[26]  Sameer Malhotra,et al.  Structured override reasons for drug-drug interaction alerts in electronic health records , 2019, J. Am. Medical Informatics Assoc..

[27]  Rachel Gold,et al.  Decrease in unnecessary vitamin D testing using clinical decision support tools: making it harder to do the wrong thing , 2017, J. Am. Medical Informatics Assoc..

[28]  Carol C Wu,et al.  Does a clinical decision rule using D-dimer level improve the yield of pulmonary CT angiography? , 2011, AJR. American journal of roentgenology.

[29]  M. Mackay,et al.  Frequency and Severity of Parenteral Nutrition Medication Errors at a Large Children's Hospital After Implementation of Electronic Ordering and Compounding. , 2016, Nutrition in clinical practice : official publication of the American Society for Parenteral and Enteral Nutrition.

[30]  Matthew W. Keefer,et al.  Improving Home Management Plan of Care Compliance Rates Through an Electronic Asthma Action Plan , 2013, The Journal of asthma : official journal of the Association for the Care of Asthma.

[31]  N. Black,et al.  The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. , 1998, Journal of epidemiology and community health.

[32]  H. Mcdonald,et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. , 2005, JAMA.

[33]  Diane L. Seger,et al.  Viewpoint Paper: Tiering Drug-Drug Interaction Alerts by Severity Increases Compliance Rates , 2009, J. Am. Medical Informatics Assoc..

[34]  Robyn Hocking,et al.  Yale MeSH Analyzer , 2017 .

[35]  Ramin Khorasani,et al.  Impact of Clinical Decision Support on Radiography for Acute Ankle Injuries: A Randomized Trial , 2017, The western journal of emergency medicine.

[36]  Tzeng-Ji Chen,et al.  Impact of a Warning CPOE System on the Inappropriate Pill Splitting of Prescribed Medications in Outpatients , 2014, PloS one.

[37]  Pamela Garcia-Filion,et al.  Computerized Dose Range Checking Using Hard and Soft Stop Alerts Reduces Prescribing Errors in a Pediatric Intensive Care Unit , 2017, Journal of patient safety.

[38]  David M Yousem.  Combating overutilization: radiology benefits managers versus order entry decision support. , 2012, Neuroimaging clinics of North America.

[39]  Judy McNulty,et al.  Methodologies for sustaining barcode medication administration compliance. A multi-disciplinary approach. , 2009, Journal of healthcare information management : JHIM.

[40]  Travis Browning,et al.  A novel use of the discrete templated notes within an electronic health record software to monitor resident supervision , 2017, J. Am. Medical Informatics Assoc..

[41]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement , 2009, BMJ.

[42]  Marc M. Triola,et al.  Brief report: Failure of an electronic medical record tool to improve pain assessment documentation , 2006, Journal of General Internal Medicine.

[43]  David W Bates,et al.  CPOE and clinical decision support in hospitals: getting the benefits: comment on "Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction". , 2010, Archives of internal medicine.

[44]  N. Ty Smith.  The Society for Technology in Anesthesia , 2005, Journal of Clinical Monitoring.

[45]  Chan Sun Park,et al.  The use of an electronic medical record system for mandatory reporting of drug hypersensitivity reactions has been shown to improve the management of patients in the university hospital in Korea , 2008, Pharmacoepidemiology and drug safety.

[46]  S. Richter,et al.  Implementation of a Clinical Decision Support Tool for Stool Cultures and Parasitological Studies in Hospitalized Patients , 2017, Journal of Clinical Microbiology.

[47]  Stephen F Spring,et al.  Evaluation of a Mandatory Quality Assurance Data Capture in Anesthesia: A Secure Electronic System to Capture Quality Assurance Information Linked to an Automated Anesthesia Record , 2011, Anesthesia and analgesia.

[48]  Diane L. Seger,et al.  Overrides of medication-related clinical decision support alerts in outpatients , 2014, J. Am. Medical Informatics Assoc..

[49]  Dean F Sittig,et al.  Clinical decision support alert appropriateness: a review and proposal for improvement. , 2014, The Ochsner journal.

[50]  Robert B. Penfold,et al.  Use of interrupted time series analysis in evaluating health care quality improvements. , 2013, Academic pediatrics.

[51]  Deborah Rose,et al.  The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age , 2018 .

[52]  Robert L Wears,et al.  Forcing functions: the need for restraint. , 2009, Annals of emergency medicine.

[53]  N Page,et al.  A systematic review of the effectiveness of interruptive medication prescribing alerts in hospital CPOE systems to change prescriber behavior and improve patient safety , 2017, Int. J. Medical Informatics.

[54]  Michael Cecchini,et al.  Electronic Intervention to Improve Structured Cancer Stage Data Capture. , 2016, Journal of oncology practice.

[55]  Andrew Goldberg,et al.  Simulation to Test Hard-Stop Implementation of a Pre-anesthetic Induction Checklist , 2014, 2014 IEEE 27th International Symposium on Computer-Based Medical Systems.

[56]  R. Griffey,et al.  Use of a computerized forcing function improves performance in ordering restraints. , 2009, Annals of emergency medicine.

[57]  M. Cantor,et al.  Implementing online medication reconciliation at a large academic medical center. , 2008, Joint Commission journal on quality and patient safety.

[58]  Nan Liu,et al.  The Effect of an Electronic "Hard-stop" Alert on HIV Testing Rates in the Emergency Department , 2013, MedInfo.

[59]  Roger B. Davis,et al.  Overrides of medication alerts in ambulatory care. , 2009, Archives of internal medicine.

[60]  Kai Zheng,et al.  Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review , 2019, J. Am. Medical Informatics Assoc..

[61]  Katja Gruenewald.  Computing Information Technology The Human Side , 2016 .

[62]  V Shalev,et al.  Mandatory computer field for blood pressure measurement improves screening. , 2005, Family practice.

[63]  George Hripcsak,et al.  Impact of electronic medication reconciliation at hospital admission on clinician workflow. , 2010, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[64]  D. Bates,et al.  Clinical Decision Support Systems , 1999, Health Informatics.

[65]  Pieter J. Helmons,et al.  Decision support at the point of prescribing to increase formulary adherence. , 2015, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[66]  Ramin Khorasani,et al.  Impact of IT-enabled intervention on MRI use for back pain. , 2014, The American journal of medicine.

[67]  Christopher DeFlitch,et al.  Mandatory Assignment of Modified Wells Score Before CT Angiography for Pulmonary Embolism Fails to Improve Utilization or Percentage of Positive Cases. , 2016, AJR. American journal of roentgenology.