Generating and Describing Affective Eye Behaviors

The manner of a person's eye movement conveys much about nonverbal information and emotional intent beyond speech. This paper describes work on expressing emotion through eye behaviors in virtual agents based on the parameters selected from the AU-Coded facial expression database and real-time eye movement data (pupil size, blink rate and saccade). A rule-based approach to generate primary (joyful, sad, angry, afraid, disgusted and surprise) and intermediate emotions (emotions that can be represented as the mixture of two primary emotions) utilized the MPEG4 FAPs (facial animation parameters) is introduced. Meanwhile, based on our research, a scripting tool, named EEMML (Emotional Eye Movement Markup Language) that enables authors to describe and generate emotional eye movement of virtual agents, is proposed.

[1]  Kostas Karpouzis,et al.  Parameterized Facial Expression Synthesis Based on MPEG-4 , 2002, EURASIP J. Adv. Signal Process..

[2]  Brent Lance,et al.  The Relation between Gaze Behavior and the Attribution of Emotion: An Empirical Study , 2008, IVA.

[3]  Yoshifumi Kitamura,et al.  Effects of avatar's blinking animation on person impressions , 2008, Graphics Interface.

[4]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[5]  Zheng Li,et al.  An Extension of MPML with Emotion Recognition Functions Attached , 2008, IVA.

[6]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[7]  Veikko Surakka,et al.  Pupil size variation as an indication of affective processing , 2003, Int. J. Hum. Comput. Stud..

[8]  M. Bradley,et al.  The pupil as a measure of emotional arousal and autonomic activation. , 2008, Psychophysiology.

[9]  Cynthia Whissell,et al.  THE DICTIONARY OF AFFECT IN LANGUAGE , 1989 .

[10]  Brent Lance,et al.  Emotionally Expressive Head and Body Movement During Gaze Shifts , 2007, IVA.

[11]  R. Plutchik.  Emotion, a psychoevolutionary synthesis , 1980 .

[12]  N. Badler,et al.  Eyes Alive Eyes Alive Eyes Alive Figure 1: Sample Images of an Animated Face with Eye Movements , 2022 .

[13]  Norman I. Badler,et al.  Visual Attention and Eye Gaze During Multiparty Conversations with Distractions , 2006, IVA.

[14]  Andreas Keil,et al.  Look–don’t look! How emotional pictures affect pro- and anti-saccades , 2008, Experimental Brain Research.

[15]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.