Automatic classification of eye activity for cognitive load measurement with emotion interference

Measuring cognitive load changes can contribute to better treatment of patients, can help design effective strategies to reduce medical errors among clinicians and can facilitate user evaluation of health care information systems. This paper proposes an eye-based automatic cognitive load measurement (CLM) system toward realizing these prospects. Three types of eye activity are investigated: pupillary response, blink and eye movement (fixation and saccade). Eye activity features are investigated in the presence of emotion interference, which is a source of undesirable variability, to determine the susceptibility of CLM systems to other factors. Results from an experiment combining arithmetic-based tasks and affective image stimuli demonstrate that arousal effects are dominated by cognitive load during task execution. To minimize the arousal effect on CLM, the choice of segments for eye-based features is examined. We then propose a feature set and classify three levels of cognitive load. The performance of cognitive load level prediction was found to be close to that of a reaction time measure, showing the feasibility of eye activity features for near-real time CLM.

[1]  N Butters,et al.  Cognitive loss and recovery in long-term alcohol abusers. , 1983, Archives of general psychiatry.

[2]  Katja Cattapan-Ludewig,et al.  Rapid Visual Information Processing in Schizophrenic Patients: The Impact of Cognitive Load and Duration of Stimulus Presentation , 2005, Neuropsychobiology.

[3]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  B. Fredrickson,et al.  Measurement issues in emotion research. , 1999 .

[5]  Jodi Forlizzi,et al.  Psycho-physiological measures for assessing cognitive load , 2010, UbiComp.

[6]  Mickaël Causse,et al.  POSITIVE AND NEGATIVE EMOTION INDUCTION THROUGH AVATARS AND ITS IMPACT ON REASONING PERFORMANCE: CARDIOVASCULAR AND PUPILLARY CORRELATES , 2012 .

[7]  Andrew T. Duchowski,et al.  Eye Tracking Methodology: Theory and Practice , 2003, Springer London.

[8]  Robert J. K. Jacob,et al.  Eye tracking in human-computer interaction and usability research : Ready to deliver the promises , 2002 .

[9]  W. Spaulding,et al.  Cognitive functioning in schizophrenia: implications for psychiatric rehabilitation. , 1999, Schizophrenia bulletin.

[10]  Minoru Nakayama,et al.  Frequency analysis of task evoked pupillary response and eye-movement , 2004, ETRA.

[11]  L. Phillips,et al.  The psychological, neurochemical and functional neuroanatomical mediators of the effects of positive and negative mood on executive functions , 2007, Neuropsychologia.

[12]  P. Maruff,et al.  Computerised cognitive assessment of athletes with sports related head injury , 2001, British journal of sports medicine.

[13]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[14]  J. Veltman,et al.  Physiological workload reactions to increasing levels of task difficulty. , 1998, Ergonomics.

[15]  J. Beatty,et al.  Pupillary responses during information processing vary with Scholastic Aptitude Test scores. , 1979, Science.

[16]  B. Tversky,et al.  Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. , 2011, Psychophysiology.

[17]  Sandra P Marshall,et al.  Identifying cognitive state from eye metrics. , 2007, Aviation, space, and environmental medicine.

[18]  Vimla L. Patel,et al.  The multitasking clinician: Decision-making and cognitive demand during and after team handoffs in emergency care , 2007, Int. J. Medical Informatics.

[19]  Siyuan Chen,et al.  Eye activity as a measure of human mental effort in HCI , 2011, IUI '11.

[20]  Itiel Dror,et al.  A novel approach to minimize error in the medical domain: Cognitive neuroscientific insights into training , 2011, Medical teacher.

[21]  M. Bradley,et al.  The pupil as a measure of emotional arousal and autonomic activation. , 2008, Psychophysiology.

[22]  R. Parasuraman,et al.  Psychophysiology and adaptive automation , 1996, Biological Psychology.

[23]  E. Ponder,et al.  ON THE ACT OF BLINKING , 1927 .

[24]  E. Tangalos,et al.  Mild Cognitive Impairment Clinical Characterization and Outcome , 1999 .

[25]  Y. Tanaka,et al.  Blink Activity and Task Difficulty , 1993, Perceptual and motor skills.

[26]  Veikko Surakka,et al.  Pupil size variation as an indication of affective processing , 2003, Int. J. Hum. Comput. Stud..

[27]  J Brender,et al.  Cognitive evaluation: how to assess the usability of information technology in healthcare. , 1997, Computer methods and programs in biomedicine.

[28]  A. Kramer,et al.  Physiological metrics of mental workload: A review of recent progress , 1990, Multiple-task performance.

[29]  Kristine Yaffe,et al.  Cognitive screening predicts magnitude of functional recovery from admission to 3 months after discharge in hospitalized elders. , 2003, The journals of gerontology. Series A, Biological sciences and medical sciences.

[30]  G Schneider,et al.  Eye tracking for assessment of workload: a pilot study in an anaesthesia simulator environment. , 2011, British journal of anaesthesia.

[31]  Tjerk de Greef,et al.  Eye Movement as Indicators of Mental Workload to Trigger Adaptive Automation , 2009, HCI.

[32]  Robert F. Stanners,et al.  The pupillary response as an indicator of arousal and cognition , 1979 .

[33]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[34]  Vered Aharonson,et al.  Human-computer interaction in the administration and analysis of neuropsychological tests , 2004, Comput. Methods Programs Biomed..

[35]  J. Beatty.  Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982 .

[36]  D. E. Irwin,et al.  Eyeblinks and cognition , 2011 .

[37]  Margrit Betke,et al.  Communication via eye blinks - detection and duration analysis in real time , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[38]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[39]  Björn Schuller,et al.  Being bored? Recognising natural interest by extensive audiovisual integration for real-life application , 2009, Image Vis. Comput..

[40]  Scott Makeig,et al.  Eye Activity Correlates of Workload during a Visuospatial Memory Task , 2001, Hum. Factors.

[41]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  A. Wingfield,et al.  Pupillometry as a measure of cognitive effort in younger and older adults. , 2010, Psychophysiology.