Usability flaws of medication-related alerting functions: A systematic qualitative review

INTRODUCTION Medication-related alerting functions may include usability flaws that limit their optimal use. A first step on the way to preventing usability flaws is to understand the characteristics of these usability flaws. This systematic qualitative review aims to analyze the type of usability flaws found in medication-related alerting functions. METHOD Papers were searched via PubMed, Scopus and Ergonomics Abstracts databases, along with references lists. Paper selection, data extraction and data analysis was performed by two to three Human Factors experts. Meaningful semantic units representing instances of usability flaws were the main data extracted. They were analyzed through qualitative methods: categorization following general usability heuristics and through an inductive process for the flaws specific to medication-related alerting functions. MAIN RESULTS From the 6380 papers initially identified, 26 met all eligibility criteria. The analysis of the papers identified a total of 168 instances of usability flaws that could be classified into 13 categories of usability flaws representing either violations of general usability principles (i.e. they could be found in any system, e.g. guidance and workload issues) or infractions specific to medication-related alerting functions. The latter refer to issues of low signal-to-noise ratio, incomplete content of alerts, transparency, presentation mode and timing, missing alert features, tasks and control distribution. MAIN CONCLUSION The list of 168 instances of usability flaws of medication-related alerting functions provides a source of knowledge for checking the usability of medication-related alerting functions during their design and evaluation process and ultimately constructs evidence-based usability design principles for these functions.

[1]  Ilkka Kunnamo,et al.  Clinical decision support must be useful, functional is not enough: a qualitative study of computer-based clinical decision support in primary care , 2012, BMC Health Services Research.

[2]  K. Thursky,et al.  Use of computerized decision support systems to improve antibiotic prescribing , 2006, Expert review of anti-infective therapy.

[3]  Monique W. M. Jaspers,et al.  Usability Studies on Interactive Health Information Systems; Where Do We Stand? , 2008, MIE.

[4]  Alissa L. Russ,et al.  Prescribers' interactions with medication alerts at the point of prescribing: A multi-method, in situ investigation of the human-computer interaction , 2012, Int. J. Medical Informatics.

[5]  Dominique L. Scapin,et al.  Ergonomic criteria for evaluating the ergonomic quality of interactive systems , 1997, Behav. Inf. Technol..

[6]  Gilad J. Kuperman,et al.  Alert Override Reasons: A Failure to Communicate , 2008, AMIA.

[7]  Mary Cooper,et al.  Impact of a computerized alert during physician order entry on medication dosing in patients with renal impairment , 2002, AMIA.

[8]  Jonathan M. Teich,et al.  Grand challenges in clinical decision support , 2008, J. Biomed. Informatics.

[9]  Christian Lovis,et al.  A Qualitative Analysis of Prescription Activity and Alert Usage in a Computerized Physician Order Entry System , 2011, MIE.

[10]  Daniel Z. Sands,et al.  Assessing the value of electronic prescribing in ambulatory care: A focus group study , 2009, Int. J. Medical Informatics.

[11]  Kaveh G Shojania,et al.  Usability evaluation of order sets in a computerised provider order entry system , 2011, BMJ quality & safety.

[12]  Mary K Goldstein,et al.  Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers. , 2010, Pain medicine.

[13]  Joan S. Ash,et al.  Some Unintended Consequences of Clinical Decision Support Systems , 2007, AMIA.

[14]  Marc Berg,et al.  Overriding of drug safety alerts in computerized physician order entry. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[15]  Emily S. Patterson,et al.  Exploring barriers and facilitators to the use of computerized clinical reminders. , 2005, Journal of the American Medical Informatics Association : JAMIA.

[16]  Tom Chan,et al.  Are we setting about improving the safety of computerised prescribing in the right way? A workshop report. , 2009, Informatics in primary care.

[17]  Nicolette de Keizer,et al.  STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics , 2009, Yearbook of Medical Informatics.

[18]  Alissa L. Russ,et al.  Computerized Medication Alerts and Prescriber Mental Models: Observing Routine Patient Care , 2009 .

[19]  Marc Berg,et al.  Viewpoint Paper: Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors , 2003, J. Am. Medical Informatics Assoc..

[20]  S Pelayo,et al.  Evolution of Human Factors Research and Studies of Health Information Technologies: The Role of Patient Safety , 2013, Yearbook of Medical Informatics.

[21]  D. Coplen.  Types of Unintended Consequences Related to Computerized Provider Order Entry , 2007 .

[22]  Davide Bolchini,et al.  A successful model and visual design for creating context-aware drug-drug interaction alerts. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[23]  David W. Bates,et al.  Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems - I-MeDeSA , 2011, J. Am. Medical Informatics Assoc..

[24]  B. Schneirdeman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[25]  E. Balas,et al.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success , 2005, BMJ : British Medical Journal.

[26]  Alissa L. Russ,et al.  The Workflow of Computerized Medication Ordering in Primary Care is Not Prescriptive , 2010 .

[27]  Douglas S. Bell,et al.  Interface design principles for usable decision support: A targeted review of best practices for clinical prescribing interventions , 2012, J. Biomed. Informatics.

[28]  R. Haynes,et al.  Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. , 1998, JAMA.

[29]  David W. Bates,et al.  Review Paper: What Evidence Supports the Use of Computerized Alerts and Prompts to Improve Clinicians' Prescribing Behavior? , 2009, J. Am. Medical Informatics Assoc..

[30]  Dean F Sittig,et al.  How to design computerized alerts to safe prescribing practices. , 2004, Joint Commission journal on quality and safety.

[31]  G. Petersson,et al.  Usability of computerised physician order entry in primary care: assessing ePrescribing with a new evaluation model. , 2011, Informatics in primary care.

[32]  Monique W. M. Jaspers,et al.  Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings , 2011, J. Am. Medical Informatics Assoc..

[33]  Farah Magrabi,et al.  Using FDA reports to inform a classification for health information technology safety problems , 2012, J. Am. Medical Informatics Assoc..

[34]  H. Mcdonald,et al.  Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. , 2005, JAMA.

[35]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[36]  A. Localio,et al.  Role of computerized physician order entry systems in facilitating medication errors. , 2005, JAMA.

[37]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[38]  Elske Ammenwerth,et al.  Seeking Evidence to Support Usability Principles for Medication-Related Clinical Decision Support (CDS) Functions , 2013, MedInfo.

[40]  S·P·M·德赖斯,et al.  Clinical Decision Support: The Road to Broad Adoption: Second Edition , 2014 .

[41]  Vimla L. Patel,et al.  Computer-based drug ordering: evaluation of interaction with a decision-support system , 2004, MedInfo.

[42]  Monique W. M. Jaspers,et al.  Effect of predefined order sets and usability problems on efficiency of computerized medication ordering , 2010, Int. J. Medical Informatics.

[43]  Peter Elkin,et al.  The Human Factors Engineering Approach to Biomedical Informatics Projects: State of the Art, Results, Benefits and Challenges , 2007, Yearbook of Medical Informatics.

[44]  Victor Nassar,et al.  Common criteria for usability review. , 2012, Work.

[45]  Martin Frischer,et al.  Barriers to the adoption of computerised decision support systems in general practice consultations: a qualitative study of GPs' perspectives , 2004, Int. J. Medical Informatics.

[46]  Marc Berg,et al.  Understanding handling of drug safety alerts: a simulation study , 2010, Int. J. Medical Informatics.

[47]  Diane L. Seger,et al.  A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems , 2010, J. Am. Medical Informatics Assoc..

[48]  J Ebbetts,et al.  Treatment of osteoporosis. , 1967, British medical journal.

[49]  Johanna I. Westbrook,et al.  The influence of computerized decision support on prescribing during ward-rounds: are the decision-makers targeted? , 2011, J. Am. Medical Informatics Assoc..

[50]  Marc Berg,et al.  Research Paper: Turning Off Frequently Overridden Drug Alerts: Limited Opportunities for Doing It Safely , 2008, J. Am. Medical Informatics Assoc..

[51]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[52]  R Khajouei,et al.  The Impact of CPOE Medication Systems’ Design Aspects on Usability, Workflow and Medication Orders , 2010, Methods of Information in Medicine.

[53]  Ornella Conort,et al.  Interventions pharmaceutiques avant et après informatisation de la prescription dans un service de médecine interne , 2007 .

[54]  Emily S. Patterson,et al.  Research Paper: Human Factors Barriers to the Effective Use of Ten HIV Clinical Reminders , 2004, J. Am. Medical Informatics Assoc..

[55]  Monique W. M. Jaspers,et al.  A Framework for reporting on Human Factor/Usability studies of Health Information Technologies , 2013, CSHI.

[56]  Alissa L. Russ,et al.  A Human Factors Investigation of Medication Alerts: Barriers to Prescriber Decision-Making and Clinical Workflow , 2009, AMIA.

[57]  Dean F. Sittig,et al.  How to Design Computerized Alerts to Ensure Safe Prescribing Practices , 2004 .

[58]  Dean F. Sittig,et al.  Computerized Provider Order Entry Adoption: Implications for Clinical Workflow , 2008, Journal of General Internal Medicine.

[59]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[60]  Dean F. Sittig,et al.  Clinician's assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements , 2002, AMIA.

[61]  Michael D Ahearn,et al.  General practitioners' perceptions of the pharmaceutical decision‐support tools in their prescribing software , 2003, The Medical journal of Australia.

[62]  Emilie M. Roth,et al.  Research Paper: Impact of Clinical Reminder Redesign on Learnability, Efficiency, Usability, and Workload for Ambulatory Clinic Nurses , 2007, J. Am. Medical Informatics Assoc..

[63]  Christian Nøhr,et al.  Four principles for user interface design of computerised clinical decision support systems. , 2011, Studies in health technology and informatics.

[64]  Alain Dauphin,et al.  [Pharmacists' interventions before and after prescription computerization in an internal medicine department]. , 2007, Presse medicale.

[65]  Vimla L. Patel,et al.  Using usability heuristics to evaluate patient safety of medical devices , 2003, J. Biomed. Informatics.

[66]  Monique W. M. Jaspers,et al.  Usability Evaluation of a Computerized Physician Order Entry for Medication Ordering , 2009, MIE.

[67]  Thomas H. Payne,et al.  Review Paper: Medication-related Clinical Decision Support in Computerized Provider Order Entry Systems: A Review , 2007, J. Am. Medical Informatics Assoc..

[68]  James D. Carpenter,et al.  Categorizing the unintended sociotechnical consequences of computerized provider order entry , 2007, Int. J. Medical Informatics.

[69]  Brian Bjørn,et al.  Computerised Physician Order Entry (CPOE). , 2009, Studies in health technology and informatics.

[70]  J. Ioannidis,et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. , 2009, Journal of clinical epidemiology.