A mixed-methods evaluation framework for electronic health records usability studies

BACKGROUND Poor EHR design adds further challenges, especially in the areas of order entry and information visualization, with a net effect of increased rates of incidents, accidents, and mortality in ICU settings. OBJECTIVE The purpose of this study was to propose a novel, mixed-methods framework to understand EHR-related information overload by identifying and characterizing areas of suboptimal usability and clinician frustration within a vendor-based, provider-facing EHR interface. METHODS A mixed-methods, live observational usability study was conducted at a single, large, tertiary academic medical center in the Southeastern US utilizing a commercial, vendor based EHR. Physicians were asked to complete usability patient cases, provide responses to three surveys, and participant in a semi-structured interview. RESULTS Of the 25 enrolled ICU physician participants, there were 5(20%) attending physicians, 9 (36%) fellows, and 11 (44%) residents; 52% of participants were females. On average, residents were the quickest in completing the tasks while attending physician took the longest to complete the same task. Poor usability, complex interface screens, and difficulty to navigate the EHR significantly correlated with high frustration levels. Significant association were found between the occurrence of error messages and temporal demand such that more error messages resulted in longer completion time (p = .03). DISCUSSION Physicians remain frustrated with the EHR due to difficulty in finding patient information. EHR usability remains a critical challenge in healthcare, with implications for medical errors, patient safety, and clinician burnout. There is a need for scientific findings on current information needs and ways to improve EHR-related information overload.

[1]  Dean F Sittig,et al.  Improving the safety of health information technology requires shared responsibility: It is time we all step up. , 2017, Healthcare.

[2]  Harold Thimbleby,et al.  Unreliable numbers: error and harm induced by bad design can be reduced by better design , 2015, Journal of The Royal Society Interface.

[3]  Bryan A. Calkin.  PARAMETERS AFFECTING MENTAL WORKLOAD AND THE NUMBER OF SIMULATED UCAVS THAT CAN BE EFFECTIVELY SUPERVISED , 2007 .

[4]  Saif S. Khairat,et al.  Perceived Burden of EHRs on Physicians at Different Stages of Their Career , 2018, Applied Clinical Informatics.

[5]  Raymond E. Levitt,et al.  The effects of CPOE on ICU workflow: an observational study , 2003, AMIA.

[6]  Michael Schaefer,et al.  Effects of different viewing perspectives on somatosensory activations during observation of touch , 2009, Human brain mapping.

[7]  Sha X. Chang,et al.  Quantification of the impact of multifaceted initiatives intended to improve operational efficiency and the safety culture: a case study from an academic medical center radiation oncology department. , 2014, Practical radiation oncology.

[8]  Vimla L. Patel,et al.  Interruptions in a level one trauma center: A case study , 2008, Int. J. Medical Informatics.

[9]  J. Beyene,et al.  Quantifying the volume of documented clinical information in critical illness. , 2008, Journal of critical care.

[10]  Rollin J Fairbanks,et al.  Electronic Health Record Vendor Adherence to Usability Certification Requirements and Testing Standards. , 2015, JAMA.

[11]  Yushi Yang,et al.  Using Eye Trackers for Usability Evaluation of Health Information Technology: A Systematic Literature Review , 2015, JMIR human factors.

[12]  Brian W Pickering,et al.  Identification of patient information corruption in the intensive care unit: Using a scoring tool to direct quality improvements in handover* , 2009, Critical care medicine.

[13]  P. Embí,et al.  Clinical decision support systems in the pediatric intensive care unit* , 2009, Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies.

[14]  Vishnu Mohan,et al.  Use of simulation to assess electronic health record safety in the intensive care unit: a pilot study , 2013, BMJ Open.

[15]  Jonathan Chan,et al.  Computer applications , 1986 .

[16]  Christine A. Sinsky,et al.  Relationship Between Clerical Burden and Characteristics of the Electronic Environment With Physician Burnout and Professional Satisfaction. , 2016, Mayo Clinic proceedings.

[17]  V. Herasevich,et al.  Novel Representation of Clinical Information in the ICU , 2010, Thrombosis and Haemostasis.

[18]  M. Privitera,et al.  Creating a "Manageable Cockpit" for Clinicians: A Shared Responsibility. , 2018, JAMA internal medicine.

[19]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[20]  Fei Yu,et al.  Toward a better understanding of task demands, workload, and performance during physician-computer interactions , 2016, J. Am. Medical Informatics Assoc..

[21]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[22]  Saif S. Khairat,et al.  Assessing the Status Quo of EHR Accessibility, Usability, and Knowledge Dissemination , 2018, EGEMS.

[23]  Martina A. Clarke,et al.  How Does Learnability of Primary Care Resident Physicians Increase After Seven Months of Using an Electronic Health Record? A Longitudinal Study , 2016, JMIR human factors.

[24]  L. Kohn,et al.  To Err Is Human : Building a Safer Health System , 2007 .