Usability Methods for Ensuring Health Information Technology Safety: Evidence-Based Approaches Contribution of the IMIA Working Group Health Informatics for Patient Safety

OBJECTIVES Issues related to lack of system usability and potential safety hazards continue to be reported in the health information technology (HIT) literature. Usability engineering methods are increasingly used to ensure improved system usability and they are also beginning to be applied more widely for ensuring the safety of HIT applications. These methods are being used in the design and implementation of many HIT systems. In this paper we describe evidence-based approaches to applying usability engineering methods. METHODS A multi-phased approach to ensuring system usability and safety in healthcare is described. Usability inspection methods are first described including the development of evidence-based safety heuristics for HIT. Laboratory-based usability testing is then conducted under artificial conditions to test if a system has any base level usability problems that need to be corrected. Usability problems that are detected are corrected and then a new phase is initiated where the system is tested under more realistic conditions using clinical simulations. This phase may involve testing the system with simulated patients. Finally, an additional phase may be conducted, involving a naturalistic study of system use under real-world clinical conditions. RESULTS The methods described have been employed in the analysis of the usability and safety of a wide range of HIT applications, including electronic health record systems, decision support systems and consumer health applications. It has been found that at least usability inspection and usability testing should be applied prior to the widespread release of HIT. However, wherever possible, additional layers of testing involving clinical simulations and a naturalistic evaluation will likely detect usability and safety issues that may not otherwise be detected prior to widespread system release. CONCLUSION The framework presented in the paper can be applied in order to develop more usable and safer HIT, based on multiple layers of evidence.

[1]  Bey-Hwa Yui,et al.  Evaluation of Computerized Physician Order Entry System—A Satisfaction Survey in Taiwan , 2012, Journal of Medical Systems.

[2]  W. Hersh,et al.  Roadmap for Provision of Safer Healthcare Information Systems: Preventing e-Iatrogenesis , 2011 .

[3]  Cathleen Wharton,et al.  Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces , 1990, CHI '90.

[4]  D. Chou.  Health IT and Patient Safety: Building Safer Systems for Better Care , 2012 .

[5]  A. Localio,et al.  Role of computerized physician order entry systems in facilitating medication errors. , 2005, JAMA.

[6]  Elizabeth M. Borycki,et al.  Designing and Integrating Clinical and Computer-based Simulations in Health Informatics: From Real-World to Virtual Reality , 2010 .

[7]  Joseph L. Kannry,et al.  Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support , 2012, Int. J. Medical Informatics.

[8]  Joseph L. Kannry,et al.  Use of Simulation Approaches in the Study of Clinician Workflow , 2006, AMIA.

[9]  J Brender,et al.  Technology-induced errors. The current use of frameworks and models from the biomedical and life sciences literatures. , 2011, Methods of information in medicine.

[10]  Nicolette de Keizer,et al.  Review Paper: Evaluation of Outpatient Computerized Physician Medication Order Entry Systems: A Systematic Review , 2007, J. Am. Medical Informatics Assoc..

[11]  Farah Magrabi,et al.  Using FDA reports to inform a classification for health information technology safety problems , 2012, J. Am. Medical Informatics Assoc..

[12]  Elizabeth M Borycki,et al.  Scenario-based testing of health information systems (HIS) in electronic and hybrid environments. , 2009, Studies in health technology and informatics.

[13]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[14]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[15]  Vimla L. Patel,et al.  Cognitive and usability engineering methods for the evaluation of clinical information systems , 2004, J. Biomed. Informatics.

[16]  Elizabeth M Borycki,et al.  Ensuring the safety of health information systems: using heuristics for patient safety. , 2009, Healthcare quarterly.

[17]  Elske Ammenwerth,et al.  Simulation Studies for the Evaluation of Health Information Technologies: Experiences and Results , 2012, Health information management : journal of the Health Information Management Association of Australia.

[18]  Andre Kushniruk,et al.  Predicting changes in workflow resulting from healthcare information systems: ensuring the safety of healthcare. , 2006, Healthcare quarterly.

[19]  Elizabeth M Borycki,et al.  Use of simulation in the study of clinician workflow. , 2006, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[20]  Elizabeth M Borycki,et al.  Technology induced error and usability: The relationship between usability problems and prescription errors when using a handheld application , 2005, Int. J. Medical Informatics.

[21]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[22]  Elizabeth M. Borycki,et al.  Using Simulation Methods to Analyze and Predict Changes in Workflow and Potential Problems in the Use of a Bar-Coding Medication Order Entry System , 2006, AMIA.

[23]  Elizabeth M. Borycki,et al.  Low-Cost Rapid Usability Testing for Health Information Systems: Is it Worth the Effort? , 2012, MIE.

[24]  Farah Magrabi,et al.  An analysis of computer-related patient safety incidents to inform the development of a classification , 2010, J. Am. Medical Informatics Assoc..

[25]  A W Kushniruk,et al.  A framework for diagnosing and identifying where technology-induced errors come from. , 2009, Studies in health technology and informatics.

[26]  Vimla L. Patel,et al.  Using usability heuristics to evaluate patient safety of medical devices , 2003, J. Biomed. Informatics.

[27]  Elizabeth M Borycki,et al.  Low-cost rapid usability engineering: designing and customizing usable healthcare information systems. , 2006, Healthcare quarterly.

[28]  Stéphanie Bernonville,et al.  Example of a Human Factors Engineering approach to a medication administration work system: Potential impact on patient safety , 2010, Int. J. Medical Informatics.

[29]  André Kushniruk,et al.  Where do Technology-Induced Errors Come From? Towards a Model for Conceptualizing and Diagnosing Errors Caused by Technology , 2008 .

[30]  Elizabeth Keay,et al.  Methods to assess the safety of health information systems. , 2010, Healthcare quarterly.

[31]  André Kushniruk,et al.  The Long and Twisting Path: An Efficiency Evaluation of an Electronic Whiteboard System , 2013, ITCH.

[32]  Dean F. Sittig,et al.  Electronic health records and national patient-safety goals. , 2012, The New England journal of medicine.