Relating Task Demand, Mental Effort and Task Difficulty with Physicians’ Performance during Interactions with Electronic Health Records (EHRs)

ABSTRACT Objective was to assess the relationship between task demand, mental effort, task difficulty, and performance during physicians’ interaction with electronic health records (EHRs). Seventeen physicians performed three EHR-based scenarios with varying task demands. Mental effort was measured using eye tracking measures via task evoked pupillary responses (TEPR), blink frequency, and gaze speed; task difficulty (or user behavior) was measured using frequent mouse click patterns and task flow; user performance was quantified using two types of omission errors: (i) omission errors with no evidence of trying to complete the task and (ii) omission error with evidence of trying but unable to complete the task. The results indicated that task demand significantly increased mental effort, but not task difficulty. Task demand, mental effort, and task difficulty all predicted performance. Specifically, there was a significant relationship between (i) task demand, TEPR and omission errors with no evidence of trying to complete the task, and (ii) blink frequency, repeated search clicks and omission error with evidence of trying but unable to complete the task. In concert, results suggest that physicians’ performance during EHR interaction was negatively affected by task demands and increase in mental effort. This highlight the need for implementation of appropriate quality assurance (QA) measures, in addition to EHR usability improvement, to minimize omission errors and improve physician’s performance. Additionally, the lack of relationship between task demand and task difficulty highlights a need for further methodological and empirical studies to advance our understanding from theory to application during physician–EHR interaction.

[1]  Jason J. Saleem,et al.  Pilot Performance, Strategy, and Workload While Executing Approaches at Steep Angles and With Lower Landing Minima , 2007, Hum. Factors.

[2]  James L. Szalma,et al.  Task Loading and Stress in Human–Computer Interaction: Theoretical Frameworks and Mitigation Strategies , 2009 .

[3]  Alan Kennedy,et al.  Book Review: Eye Tracking: A Comprehensive Guide to Methods and Measures , 2016, Quarterly journal of experimental psychology.

[4]  François Sainfort,et al.  Evaluating usability of a commercial electronic health record: A case study , 2008, Int. J. Hum. Comput. Stud..

[5]  D. Kahneman,et al.  Attention and Effort , 1973 .

[6]  Monique W. M. Jaspers,et al.  The significance of a usability evaluation of an emerging laboratory order entry system , 2007, Int. J. Medical Informatics.

[7]  P A Hancock,et al.  Influence of task demand characteristics on workload and performance. , 1995, The International journal of aviation psychology.

[8]  Ravi S. Adapathya,et al.  Strategic Behavior, Workload, and Performance in Task Scheduling , 1991 .

[9]  C. Collet,et al.  Complexity and Controller Workload : Trying to Bridge the Gap , 2002 .

[10]  Sheue-Ling Hwang,et al.  Evaluation and prediction of on-line maintenance workload in nuclear power plants , 2009 .

[11]  Barry H. Kantowitz,et al.  Mental Workload , 2020, Encyclopedia of Behavioral Medicine.

[12]  Robert G Hill,et al.  4000 clicks: a productivity analysis of electronic medical records in a community hospital ED. , 2013, The American journal of emergency medicine.

[13]  Eric Hoffman,et al.  Introducing a New Spacing Instruction. Impact of Spacing Tolerance on Flight Crew Activity , 2003 .

[14]  Barry H. Kantowitz,et al.  3. Mental Workload , 1987 .

[15]  Peter A. Hancock,et al.  ADAPTIVE CONTROL OF MENTAL WORKLOAD. , 2001 .

[16]  David H. Laidlaw,et al.  A Case Study Using Visualization Interaction Logs and Insight Metrics to Understand How Analysts Arrive at Insights , 2016, IEEE Transactions on Visualization and Computer Graphics.

[17]  L. Mulder,et al.  Please Scroll down for Article Ergonomics Cardiovascular and Eye Activity Measures as Indices for Momentary Changes in Mental Effort during Simulated Flight Cardiovascular and Eye Activity Measures as Indices for Momentary Changes in Mental Effort during Simulated Flight , 2022 .

[18]  Brian Hilburn,et al.  COGNITIVE COMPLEXITY IN AIR TRAFFIC CONTROL: A LITERATURE REVIEW , 2004 .

[19]  L GoodhueDale,et al.  Task-technology fit and individual performance , 1995 .

[20]  Jaime Arguello,et al.  The effect of cognitive abilities on information search for tasks of varying levels of complexity , 2014, IIiX.

[21]  Shuk Ying Ho,et al.  Human-computer interaction and management information systems: Foundations , 2008, J. Assoc. Inf. Sci. Technol..

[22]  Dipak Kalra,et al.  How do clinical information systems affect the cognitive demands of general practitioners? Usability study with a focus on cognitive workload , 2015, BMJ Health & Care Informatics.

[23]  D. Johnston,et al.  Electronic Health Record–Related Events in Medical Malpractice Claims , 2015, Journal of patient safety.

[24]  N Moray,et al.  Where is capacity limited? A survey and a model. , 1967, Acta psychologica.

[25]  Fei Yu,et al.  Toward a better understanding of task demands, workload, and performance during physician-computer interactions , 2016, J. Am. Medical Informatics Assoc..

[26]  Kai Zheng,et al.  Research Article: An Interface-driven Analysis of User Interactions with an Electronic Health Records System , 2009, J. Am. Medical Informatics Assoc..

[27]  Glenn F. Wilson,et al.  Psychophysiological responses to changes in workload during simulated air traffic control , 1996, Biological Psychology.

[28]  Linden J. Ball,et al.  Eye tracking in HCI and usability research. , 2006 .

[29]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[30]  Natasha Merat,et al.  Comparing the driving performance of average and older drivers: The effect of surrogate in-vehicle information systems , 2005 .

[31]  Arthur D. Fisk,et al.  The Effects of Experience and Strategy on Visual Attention Allocation in an Automated Multiple-Task Environment , 2014, Int. J. Hum. Comput. Interact..

[32]  Rema Padman,et al.  Paving the COWpath: data-driven design of pediatric order sets. , 2014, Journal of the American Medical Informatics Association : JAMIA.

[33]  J. C. Spearandio Variation of operator's strategies and regulating effects on workload. , 1971, Ergonomics.

[34]  J. May,et al.  Eye movement indices of mental workload. , 1990, Acta psychologica.

[35]  Herbert A. Colle,et al.  Estimating a Mental Workload Redline in a Simulated Air-to-Ground Combat Mission , 2005 .

[36]  James L. Szalma,et al.  Operator Stress and Display Design , 2003 .

[37]  Erin Sparnon,et al.  The Role of the Electronic Health Record in Patient Safety Events , 2012 .

[38]  Paul Kolm,et al.  Effect of hospitalist workload on the quality and efficiency of care. , 2014, JAMA internal medicine.

[39]  Charlotte A. Weaver,et al.  Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. , 2013, Journal of the American Medical Informatics Association : JAMIA.

[40]  E. Cohen,et al.  Off the Record — Avoiding the Pitfalls of Going Electronic , 2009 .

[41]  Louise Venables,et al.  The influence of task demand and learning on the psychophysiological response. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[42]  Lawrence B. Marks,et al.  Usability Evaluation of Electronic Health Record System (EHRs) using Subjective and Objective Measures , 2016, CHIIR.

[43]  K. J. Vicente,et al.  Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work , 1999 .

[44]  Pamela Stetina,et al.  Managing medication errors--a qualitative study. , 2005, Medsurg nursing : official journal of the Academy of Medical-Surgical Nurses.

[45]  James L. Szalma,et al.  Individual Differences in the Stress and Workload of Sustained Attention , 2002 .

[46]  G. Young,et al.  Assessment of workload using NASA Task Load Index in perianesthesia nursing. , 2008, Journal of perianesthesia nursing : official journal of the American Society of PeriAnesthesia Nurses.

[47]  Jonathan Histon The impact of structure on cognitive complexity in air traffic control , 2002 .

[48]  Prithima R Mosaly,et al.  Quantifying the impact of cross coverage on physician's workload and performance in radiation oncology. , 2013, Practical radiation oncology.

[49]  Ma Sasse,et al.  The Eyes Never Lie: The Use of Eyetracking Data in HCI Research , 2002, CHI 2002.

[50]  W. B. Seales,et al.  Assessing Mental Workload During Laparoscopic Surgery , 2005, Surgical innovation.

[51]  Fátima Pereira da Silva Mental Workload, Task Demand and Driving Performance: What Relation?☆ , 2014 .

[52]  Shayne Loft,et al.  Modeling and Predicting Mental Workload in En Route Air Traffic Control: Critical Review and Broader Implications , 2007, Hum. Factors.

[53]  Seong Nam Byun,et al.  An Evaluation of the Operator Mental Workload of Advanced Control Facilities in Korea Next Generation Reactor , 2002 .

[54]  M. Stella Atkins,et al.  Workload assessment of surgeons: correlation between NASA TLX and blinks , 2012, Surgical Endoscopy.

[55]  Morten Hertzum,et al.  Perceived Time as a Measure of Mental Workload: Effects of Time Constraints and Task Success , 2013, Int. J. Hum. Comput. Interact..

[56]  Jiajie Zhang,et al.  TURF: Toward a unified framework of EHR usability , 2011, J. Biomed. Informatics.

[57]  Siyuan Chen,et al.  Eye activity as a measure of human mental effort in HCI , 2011, IUI '11.

[58]  Dimitrios Stefanidis,et al.  Higher Mental Workload is Associated With Poorer Laparoscopic Performance as Measured by the NASA-TLX Tool , 2010, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[59]  Katharine K. Lee,et al.  Development and Validation of the Controller Acceptance Rating Scale (CARS): Results of Empirical Research , 2001 .

[60]  Rema Padman,et al.  Reducing Provider Cognitive Workload in CPOE Use: Optimizing Order Sets , 2013, MedInfo.

[61]  Tamsyn Edwards,et al.  Task Demand Variation in Air Traffic Control: Implications for Workload, Fatigue, and Performance , 2017 .

[62]  Alissa L. Russ,et al.  Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. , 2014, Journal of the American Medical Informatics Association : JAMIA.

[63]  Sue E Bowman Impact of electronic health record systems on information integrity: quality and safety implications. , 2013, Perspectives in health information management.

[64]  J. Beatty,et al.  The pupillary system. , 2000 .

[65]  Herbert S. Lin,et al.  Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions , 2009 .

[66]  Gregg Tracton,et al.  Improving radiation oncology providers' workload and performance: Can simulation-based training help? , 2017, Practical radiation oncology.

[67]  W. Baine,et al.  The Agency for Healthcare Research and Quality , 2006, Italian Journal of Public Health.

[68]  D. E. Broadbent CHAPTER 10 – THE SELECTIVE NATURE OF LEARNING , 1958 .

[69]  Hasan Ayaz,et al.  Cognitive Workload Assessment of Air Traffic Controllers Using Optical Brain Imaging Sensors , 2010 .