Exploring Eye Activity as an Indication of Emotional States Using an Eye-Tracking Sensor

The automatic detection of human emotional states has been of great interest lately for its applications not only in the Human-Computer Interaction field, but also for its applications in psychological studies. Using an emotion elicitation paradigm, we investigate whether eye activity holds discriminative power for detecting affective states. Our emotion elicitation paradigm includes induced emotions by watching emotional movie clips and spontaneous emotions elicited by interviewing participants about emotional events in their life. To reduce gender variability, the selected participants were 60 female native Arabic speakers (30 young adults, and 30 mature adults). In general, the automatic classification results using eye activity were reasonable, giving 66% correct recognition rate on average. Statistical measures show statistically significant differences in eye activity patterns between positive and negative emotions. We conclude that eye activity, including eye movement, pupil dilation and pupil invisibility could be used as a complementary cues for the automatic recognition of human emotional states.

[1]  Richard J. Davidson,et al.  Now You Feel It, Now You Don't , 2003, Psychological science.

[2]  Feng Zhou,et al.  Affective and cognitive design for mass personalization: status and prospect , 2012, Journal of Intelligent Manufacturing.

[3]  Scotty D. Craig,et al.  Affect and learning: An exploratory look into the role of affect in learning with AutoTutor , 2004 .

[4]  D. Kahneman,et al.  Pupillary, heart rate, and skin resistance changes during a mental task. , 1969, Journal of experimental psychology.

[5]  S. Sirois,et al.  Pupillometry , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[6]  R. Davidson,et al.  Manipulating affective state using extended picture presentations. , 1997, Psychophysiology.

[7]  Veikko Surakka,et al.  Pupillary responses to emotionally provocative stimuli , 2000, ETRA.

[8]  Wen-Bing Horng,et al.  Driver fatigue detection based on eye tracking and dynamk, template matching , 2004, IEEE International Conference on Networking, Sensing and Control, 2004.

[9]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[10]  M. Bradley,et al.  The pupil as a measure of emotional arousal and autonomic activation. , 2008, Psychophysiology.

[11]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[12]  Daniel M. Russell,et al.  Discriminating the relevance of web search results with measures of pupil size , 2009, CHI.

[13]  R. P. Hobson,et al.  Emotion recognition in autism: coordinating faces and voices , 1988, Psychological Medicine.

[14]  Tamás D. Gedeon,et al.  A comparative study of different classifiers for detecting depression from spontaneous speech , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[15]  J. Stainer,et al.  The Emotions , 1922, Nature.

[16]  Ilia Uma.  physiological signals based human emotion recognition a review , 2014 .

[17]  Alex Pentland,et al.  Human computing and machine understanding of human behavior: a survey , 2006, ICMI '06.

[18]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[19]  Veikko Surakka,et al.  Pupil size variation as an indication of affective processing , 2003, Int. J. Hum. Comput. Stud..

[20]  R. Larsen,et al.  Affect intensity as an individual difference characteristic: A review , 1987 .

[21]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[22]  Michael Wagner,et al.  From Joyous to Clinically Depressed: Mood Detection Using Spontaneous Speech , 2012, FLAIRS.

[23]  J. Gross,et al.  Emotion elicitation using films , 1995 .

[24]  B. Goldwater.  Psychological significance of pupillary movements. , 1972, Psychological bulletin.

[25]  J. Beatty.  Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982 .

[26]  Tieniu Tan,et al.  Affective Computing: A Review , 2005, ACII.

[27]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  M. Argyle,et al.  EYE-CONTACT, DISTANCE AND AFFILIATION. , 1965, Sociometry.