Eye tracking cognitive load using pupil diameter and microsaccades with fixed gaze

Pupil diameter and microsaccades are captured by an eye tracker and compared for their suitability as indicators of cognitive load (as beset by task difficulty). Specifically, two metrics are tested in response to task difficulty: (1) the change in pupil diameter with respect to inter- or intra-trial baseline, and (2) the rate and magnitude of microsaccades. Participants performed easy and difficult mental arithmetic tasks while fixating a central target. Inter-trial change in pupil diameter and microsaccade magnitude appear to adequately discriminate task difficulty, and hence cognitive load, if the implied causality can be assumed. This paper’s contribution corroborates previous work concerning microsaccade magnitude and extends this work by directly comparing microsaccade metrics to pupillometric measures. To our knowledge this is the first study to compare the reliability and sensitivity of task-evoked pupillary and microsaccadic measures of cognitive load.

[1]  Michael B. McCamy,et al.  Microsaccade and drift dynamics reflect mental fatigue , 2013, The European journal of neuroscience.

[2]  M. Turatto,et al.  Visual oddballs induce prolonged microsaccadic inhibition , 2007, Experimental Brain Research.

[3]  Hunter A. Murphy,et al.  3-D eye movement analysis , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[4]  Giulio Jacucci,et al.  The Psychophysiology Primer: A Guide to Methods and a Broad Review with a Focus on Human-Computer Interaction , 2016, Found. Trends Hum. Comput. Interact..

[5]  Vsevolod Peysakhovich.  Study of pupil diameter and eye movements to enhance flight safety. Etude de diamètre pupillaire et de mouvements oculaires pour la sécurité aérienne , 2016 .

[6]  E. Hess,et al.  Pupil Size in Relation to Mental Activity during Simple Problem-Solving , 1964, Science.

[7]  Colin W. G. Clifford,et al.  Corrections to: gaze categorization under uncertainty: psychophysics and modeling , 2013 .

[8]  M. A. Frens,et al.  Recording eye movements with video-oculography and scleral search coils: a direct comparison of two methods , 2002, Journal of Neuroscience Methods.

[9]  J. Beatty,et al.  The pupillary system. , 2000 .

[10]  Francisco M. Costela,et al.  Task difficulty in mental arithmetic affects microsaccadic rates and magnitudes , 2014, The European journal of neuroscience.

[11]  Jan-Louis Kruger,et al.  Measuring the impact of subtitles on cognitive load: eye tracking and dynamic audiovisual texts , 2013, ETSA '13.

[12]  Pat Hanrahan,et al.  Measuring the task-evoked pupillary response with a remote eye tracker , 2008, ETRA.

[13]  Gaoxing Mei,et al.  The timescale of adaptation at early and mid-level stages of visual processing. , 2017, Journal of vision.

[14]  T. Watson,et al.  Spatial compression: Dissociable effects at the time of saccades and blinks. , 2015, Journal of vision.

[15]  H. Keselman,et al.  Consequences of Assumption Violations Revisited: A Quantitative Review of Alternatives to the One-Way Analysis of Variance F Test , 1996 .

[16]  E. Yund,et al.  Improving digit span assessment of short-term verbal memory , 2011, Journal of clinical and experimental neuropsychology.

[17]  Daniel Afergan,et al.  Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State , 2016, CHI.

[18]  G. Woodman,et al.  The neurophysiological index of visual working memory maintenance is not due to load dependent eye movements , 2014, Neuropsychologia.

[19]  J. Beatty.  Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982, Psychological bulletin.

[20]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[21]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[22]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[23]  Ankit Mathur,et al.  Pupil shape as viewed along the horizontal visual field. , 2013, Journal of vision.

[24]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[25]  E. Donchin,et al.  Localization of the event-related potential novelty response as defined by principal components analysis. , 2003, Brain research. Cognitive brain research.

[26]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[27]  H. Simon,et al.  Motivational and emotional controls of cognition. , 1967, Psychological review.

[28]  M. Stella Atkins,et al.  Pupil responses during discrete goal-directed movements , 2014, CHI.

[29]  R. Engle,et al.  Executive Attention, Working Memory Capacity, and a Two-Factor Theory of Cognitive Control. , 2003 .

[30]  Eakta Jain,et al.  Decoupling light reflex from pupillary dilation to measure emotional arousal in videos , 2016, SAP.

[31]  Brian P. Bailey,et al.  Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management , 2008, TCHI.

[32]  Hong-Jin Sun,et al.  Modulation of microsaccade rate by task difficulty revealed through between- and within-trial comparisons. , 2015, Journal of vision.

[33]  Cristian Hofmann,et al.  Integrating cognitive load theory and concepts of human-computer interaction , 2010, Comput. Hum. Behav..

[34]  Mark Guzdial,et al.  Measuring cognitive load in introductory CS: adaptation of an instrument , 2014, ICER '14.

[35]  Giovanni Galfano,et al.  Working memory load modulates microsaccadic rate. , 2017, Journal of vision.

[36]  Francisco M. Costela,et al.  Task difficulty in mental arithmetic affects microsaccadic rates and magnitudes , 2014 .

[37]  Robert Grover Brown,et al.  Introduction to random signal analysis and Kalman filtering , 1983 .

[38]  D. Kahneman.  Attention and Effort , 1973 .

[39]  Martin Raubal,et al.  Measuring Cognitive Load for Map Tasks Through Pupil Diameter , 2016, GIScience.

[40]  G. McCarthy,et al.  Dissociable prefrontal brain systems for attention and emotion , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[41]  Fang Chen,et al.  Designing Cognition-Adaptive Human-Computer Interface for Mission-Critical Systems , 2008, ISD.

[42]  B. Tversky,et al.  Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. , 2011, Psychophysiology.

[43]  E. Granholm,et al.  Pupillary responses index cognitive resource limitations. , 1996, Psychophysiology.

[44]  R. Ornstein,et al.  Pupillary Responses During Information Processing Vary with Scholastic Aptitude Test Scores , 2022 .

[45]  Marcus Nyström,et al.  An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data , 2010, Behavior research methods.

[46]  Xoana G. Troncoso,et al.  Saccades and microsaccades during visual fixation, exploration, and search: foundations for a common saccadic generator. , 2008, Journal of vision.

[47]  Siyuan Chen,et al.  Using Task-Induced Pupil Diameter and Blink Rate to Infer Cognitive Load , 2014, Hum. Comput. Interact..

[48]  L. Stark,et al.  The main sequence, a tool for studying human eye movements , 1975 .

[49]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[50]  D. Hubel,et al.  The role of fixational eye movements in visual perception , 2004, Nature Reviews Neuroscience.

[51]  Emanuel Schmider,et al.  Is It Really Robust , 2010 .

[52]  Ralf Engbert,et al.  Microsaccades uncover the orientation of covert attention , 2003, Vision Research.

[53]  Sharon L. Oviatt,et al.  Human-centered design meets cognitive load theory: designing interfaces that help people think , 2006, MM '06.

[54]  Ralf Engbert.  Microsaccades: A microcosm for research on oculomotor control, attention, and visual perception. , 2006, Progress in brain research.

[55]  Taylor R. Hayes,et al.  Mapping and correcting the influence of gaze position on pupil size measurements , 2015, Behavior Research Methods.

[56]  H. van Steenbergen,et al.  Pupil dilation as an index of effort in cognitive control tasks: A review , 2018, Psychonomic Bulletin & Review.

[57]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[58]  A. Wingfield,et al.  Pupillometry as a measure of cognitive effort in younger and older adults. , 2010, Psychophysiology.

[59]  R. Engle.  Working Memory Capacity as Executive Attention , 2002 .

[60]  J Hyönä,et al.  Pupil Dilation as a Measure of Processing Load in Simultaneous Interpretation and Other Language Tasks , 1995, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[61]  P. A. Gorry.  General least-squares smoothing and differentiation by the convolution (Savitzky-Golay) method , 1990 .