The Use of Task-Evoked Pupillary Response as an Objective Measure of Cognitive Load in Novices and Trained Physicians: A New Tool for the Assessment of Expertise

Purpose Task-evoked pupillary responses (TEPRs), or changes in pupil size, correlate with changes in cognitive processing demands. The magnitude of this change is a reliable marker of cognitive load. The authors used TEPRs to compare cognitive load between novices and trained physicians as they answered clinical knowledge questions. Method In 2013, 20 emergency medicine trainees were recruited and divided into novice (n = 10) and trained physician (n = 10) groups. The authors used mobile eye-tracking glasses to assess changes in pupil diameter as participants answered arithmetic questions, general knowledge questions, and clinical emergency medicine questions in a controlled setting. Questions were categorized by difficulty a priori. Results Difficult arithmetic questions caused greater changes in TEPRs than easy ones (P = .024). TEPRs were similar between groups when answering general knowledge questions (P = .383) but were significantly greater for novices than trained physicians when answering clinical questions (P < .001). TEPRs in trained physicians were significantly greater when answering difficult clinical questions than easy ones (P < .001), whereas TEPRs in novices were similar (P = .291). For those clinical questions answered correctly by both groups, TEPRs in novices were greater than those in trained physicians despite all participants answering correctly (P < .001). Conclusions Novices require more mental effort to answer clinical questions than trained physicians, even when both respond correctly. Measuring TEPRs has the potential to be a valuable assessment tool by providing objective measures of expertise and is worthy of further study.

[1]  Daniel Howes,et al.  Increasing Pupil Size Is Associated with Increasing Cognitive Processing Demands: A Pilot Study Using a Mobile Eye-Tracking Device , 2014 .

[2]  Anna Siri,et al.  The Use Of Item Analysis For The Improvement Of Objective Examinations , 2011 .

[3]  Francis J. DeMatteo,et al.  Human Learning , 2020, Delivering Psycho-educational Evaluation Results to Parents.

[4]  Pat Hanrahan,et al.  Measuring the task-evoked pupillary response with a remote eye tracker , 2008, ETRA.

[5]  E. Hess ATTITUDE AND PUPIL SIZE. , 1965, Scientific American.

[6]  B. Tversky,et al.  Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. , 2011, Psychophysiology.

[7]  D. Howes,et al.  Combining First-Person Video and Gaze-Tracking in Medical Simulation: A Technical Feasibility Study , 2014, TheScientificWorldJournal.

[8]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[9]  Krista E. DeLeeuw,et al.  A Comparison of Three Measures of Cognitive Load: Evidence for Separable Measures of Intrinsic, Extraneous, and Germane Load , 2008 .

[10]  E. Hess,et al.  Pupil Size in Relation to Mental Activity during Simple Problem-Solving , 1964, Science.

[11]  K. A. Ericsson,et al.  Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[12]  John R. Wilson,et al.  The nature of expertise: a review. , 2006, Applied ergonomics.

[13]  T. Jong Cognitive load theory, educational research, and instructional design: some food for thought , 2010 .

[14]  Joseph T. McGuire,et al.  Decision making and the avoidance of cognitive demand. , 2010, Journal of experimental psychology. General.

[15]  W. A. Mehrens,et al.  USING ITEM ANALYSIS TO IMPROVE TESTS , 1967 .

[16]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[17]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982, Psychological bulletin.

[18]  N. McGlynn Thinking fast and slow. , 2014, Australian veterinary journal.

[19]  Beth Crandall,et al.  Characterizing novice-expert differences in macrocognition: an exploratory study of cognitive work in the emergency department. , 2013, Annals of emergency medicine.

[20]  Marsha C. Lovett,et al.  How learning works , 2010 .

[21]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[22]  K. A. Ericsson,et al.  The making of an expert. , 2007, Harvard business review.

[23]  Sylvain Sirois,et al.  Pupil diameter measurement errors as a function of gaze direction in corneal reflection eyetrackers , 2013, Behavior research methods.

[24]  R. Säljö,et al.  Expertise Differences in the Comprehension of Visualizations: a Meta-Analysis of Eye-Tracking Research in Professional Domains , 2011 .

[25]  K. A. Ericsson,et al.  Long-term working memory. , 1995, Psychological review.