Usability Evaluation of Electronic Health Record System (EHRs) using Subjective and Objective Measures

The objective of this pilot study was to explore the applicability of various evaluation methods (subjective and objective) for assessing usability of electronic health record system (EHRs) during physician interaction on simple vs. complex tasks. Five physicians performed two simulated clinical scenarios consisting of 9 tasks using the EHR. Tasks were categorized into simple vs. complex tasks based on the user, task and context characteristics by the subject matter expert. Usability was assessed using four methods, (1) subjectively using subject's informal feedback and usability expert's heuristics, (2) workload measures using eye tracking, (3) behavior measures using clicks and navigation windows, and (4) performance measures using actual time on task and predictive time based on CogTool ©. Overall, the results suggest that heuristic methods (1) are highly effective in identifying usability issues, with other methods (2-4) providing complementary analysis to identify differences is task complexly and user experience with EHRs.

[1]  J. Beatty,et al.  The pupillary system. , 2000 .

[2]  Herbert S. Lin,et al.  Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions , 2009 .

[3]  Robert C. Williges,et al.  Criteria For Evaluating Usability Evaluation Methods , 2001, Int. J. Hum. Comput. Interact..

[4]  N. Schmitt.  Formulaic Sequences: Acquisition, Processing and Use , 2004 .

[5]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[6]  Mark A Frye,et al.  The eyes have it , 2017, Nature Physics.

[7]  Hans Supèr,et al.  Difference in Visual Processing Assessed by Eye Vergence Movements , 2013, PloS one.

[8]  Prithima R Mosaly,et al.  Relating physician's workload with errors during radiation therapy planning. , 2014, Practical radiation oncology.

[9]  G. Underwood,et al.  The eyes have it: An eye-movement study into the processing of formulaic sequences , 2004 .

[10]  D. Chou.  Health IT and Patient Safety: Building Safer Systems for Better Care , 2012 .

[11]  Monique W. M. Jaspers,et al.  A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence , 2009, Int. J. Medical Informatics.

[12]  Richard C. Hicks,et al.  Task Complexity and Informing Science: A Synthesis , 2006, Informing Sci. Int. J. an Emerg. Transdiscipl..

[13]  Erin Sparnon,et al.  The Role of the Electronic Health Record in Patient Safety Events , 2012 .

[14]  Jiajie Zhang,et al.  TURF: Toward a unified framework of EHR usability , 2011, J. Biomed. Informatics.

[15]  Electronic Health Record Usability Interface Design Considerations , 2009 .