A Model Relating Pupil Diameter to Mental Workload and Lighting Conditions

In this paper, we present a proof-of-concept approach to estimating mental workload by measuring the user's pupil diameter under various controlled lighting conditions. Knowing the user's mental workload is desirable for many application scenarios, ranging from driving a car, to adaptive workplace setups. Typically, physiological sensors allow inferring mental workload, but these sensors might be rather uncomfortable to wear. Measuring pupil diameter through remote eye-tracking instead is an unobtrusive method. However, a practical eye-tracking-based system must also account for pupil changes due to variable lighting conditions. Based on the results of a study with tasks of varying mental demand and six different lighting conditions, we built a simple model that is able to infer the workload independently of the lighting condition in 75% of the tested conditions.

[1]  Bryan Reimer,et al.  Sensitivity of Physiological Measures for Detecting Systematic Variations in Cognitive Demand From a Working Memory Task , 2012, Hum. Factors.

[2]  Arnaud Delorme,et al.  Frontal midline EEG dynamics during working memory , 2005, NeuroImage.

[3]  Craig K. Enders,et al.  An introduction to modern missing data analyses. , 2010, Journal of school psychology.

[4]  Ying Wang,et al.  The Impact of Repeated Cognitive Tasks on Driving Performance and Visual Attention , 2010 .

[5]  W. Kirchner.  Age differences in short-term retention of rapidly changing information. , 1958, Journal of experimental psychology.

[6]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[7]  Andrew L. Kun,et al.  Using speech, GUIs and buttons in police vehicles: field data on user preferences for the Project54 system , 2013, AutomotiveUI.

[8]  Albrecht Schmidt,et al.  Exploring user expectations for context and road video sharing while calling and driving , 2013, AutomotiveUI.

[9]  M. A. Recarte,et al.  COGNITIVE DEMANDS OF HANDS-FREE-PHONE CONVERSATION WHILE DRIVING , 2002 .

[10]  J. Dusek,et al.  Impact of Incremental Increases in Cognitive Workload on Physiological Arousal and Performance in Young Adult Drivers , 2009 .

[11]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[12]  Zhiwei Zhu,et al.  Real-time eye detection and tracking under various light conditions , 2002, ETRA.

[13]  Scott Makeig,et al.  Eye Activity Correlates of Workload during a Visuospatial Memory Task , 2001, Hum. Factors.

[14]  Kilseop Ryu,et al.  Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic , 2005 .

[15]  J. Stern,et al.  Eye movements and blinks: their relationship to higher cognitive processes. , 1989, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[16]  Andrew L. Kun,et al.  Exploring the effects of visual cognitive load and illumination on pupil diameter in driving simulators , 2012, ETRA.

[17]  T. Jung,et al.  Task performance and eye activity: predicting behavior relating to cognitive workload. , 2007, Aviation, space, and environmental medicine.

[18]  D. Kahneman,et al.  Effects of Grouping on the Pupillary Response in a Short-Term Memory Task , 1968, The Quarterly journal of experimental psychology.

[19]  C. Antonya,et al.  EYE TRACKING APPLICATIONS , 2009 .

[20]  A. H. Roscoe.  Assessing pilot workload. Why measure heart rate, HRV and respiration? , 1992, Biological Psychology.

[21]  Albrecht Schmidt,et al.  Shifting Gears: User Interfaces in the Age of Autonomous Driving , 2016, IEEE Pervasive Computing.

[22]  Ulf Ahlstrom,et al.  Using eye movement activity as a correlate of cognitive workload , 2006 .

[23]  Georg Brügner,et al.  Occupational stress and strain of female students: results of physiological, behavioral, and psychological monitoring , 1996, Biological Psychology.

[24]  William F. Moroney,et al.  A comparison of two scoring procedures with the NASA task load index in a simulated flight task , 1992, Proceedings of the IEEE 1992 National Aerospace and Electronics Conference@m_NAECON 1992.

[25]  Bruce N. Walker,et al.  Comparing heart rate and pupil size as objective measures of workload in the driving context: initial look , 2015, AutomotiveUI.

[26]  J. Beatty.  Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982, Psychological bulletin.

[27]  D. Gronwall.  Paced Auditory Serial-Addition Task: A Measure of Recovery from Concussion , 1977, Perceptual and motor skills.

[28]  Stephen A Burns,et al.  Pupil location under mesopic, photopic, and pharmacologically dilated conditions. , 2002, Investigative ophthalmology & visual science.

[29]  Andrew L. Kun,et al.  Video call, or not, that is the question , 2012, CHI EA '12.

[30]  Michael E. Smith,et al.  Neurophysiological measures of cognitive workload during human-computer interaction , 2003 .

[31]  Robert J. K. Jacob,et al.  Designing a passive brain computer interface using real time classification of functional near-infrared spectroscopy , 2013, Int. J. Auton. Adapt. Commun. Syst..

[32]  Jean Vanderdonckt,et al.  Toward Usable Intelligent User Interface , 2015, HCI.

[33]  Jonathan Dobres,et al.  A Field Study Assessing Driving Performance, Visual Attention, Heart Rate and Subjective Ratings in Response to Two Types of Cognitive Workload , 2017 .

[34]  E. Salas,et al.  Human Factors : The Journal of the Human Factors and Ergonomics Society , 2012 .

[35]  Robert Oostenveld,et al.  Estimating workload using EEG spectral power and ERPs in the n-back task , 2012, Journal of neural engineering.

[36]  Matthias Scheutz,et al.  Brainput: enhancing interactive systems with streaming fnirs brain input , 2012, CHI.

[37]  Simone Benedetto,et al.  Driver workload and eye blink duration , 2011 .

[38]  Andrew L. Kun,et al.  Estimating cognitive load using remote eye tracking in a driving simulator , 2010, ETRA.

[39]  Kristina Höök,et al.  CHI '12 Extended Abstracts on Human Factors in Computing Systems , 2012, CHI 2012.

[40]  Bryan Reimer,et al.  Driver wellness, safety and the development of an AwareCar , 2009 .

[41]  B. Tversky,et al.  Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. , 2011, Psychophysiology.

[42]  Pat Hanrahan,et al.  Measuring the task-evoked pupillary response with a remote eye tracker , 2008, ETRA.

[43]  Andrew L. Kun,et al.  Exploring the effects of size and luminance of visual targets on the pupillary light reflex , 2012, AutomotiveUI.

[44]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[45]  Andrew L. Kun,et al.  Exploring the Influence of Light and Cognitive Load on Pupil Diameter Driving Simulator Studies , 2017 .

[46]  S. P. Marshall,et al.  The Index of Cognitive Activity: measuring cognitive workload , 2002, Proceedings of the IEEE 7th Conference on Human Factors and Power Plants.

[47]  Bryan Reimer,et al.  An on-road assessment of the impact of cognitive workload on physiological arousal in young adult drivers , 2009, AutomotiveUI.

[48]  Andrew L. Kun,et al.  Estimating cognitive load using pupil diameter during a spoken dialogue task , 2013, AutomotiveUI.

[49]  Bastian Pfleging.  (Non-) Driving-Related Activities in the Car: Defining Driver Activities for Manual and Automated Driving , 2015 .

[50]  John L. Sibert,et al.  Heart rate variability: indicator of user state as an aid to human-computer interaction , 1998, CHI.

[51]  M. Tscheligi,et al.  Experiencing Autonomous Vehicles: Crossing the Boundaries between a Drive and a Ride , 2015, CHI Extended Abstracts.

[52]  Andrew L. Kun,et al.  On the feasibility of using pupil diameter to estimate cognitive load changes for in-vehicle spoken dialogues , 2013, INTERSPEECH.

[53]  Stefan Schneegaß,et al.  A data set of real world driving to assess driver workload , 2013, AutomotiveUI.