The Look of Fear from the Eyes Varies with the Dynamic Sequence of Facial Actions

Most research on the ability to interpret expressions from the eyes has utilized static information. This research investigates whether the dynamic sequence of facial actions in the eye region influences the judgments of perceivers. Dynamic fear expressions involving the eye region and eyebrows were created which systematically differed in the sequential occurrence of facial actions. Participants rated the intensity of sequential fear expressions, either in addition to a simultaneous, full-blown expression (Experiment 1) or in combination with different levels of eye gaze (Experiment 2). The results showed that the degree of attributed emotion and the appraisal ratings differed as a function of the sequence of facial expressions of fear, with direct gaze resulting in stronger subjective responses. The findings challenge current notions surrounding the study of static facial displays from the eyes and suggest that emotion perception is a dynamic process shaped by the time course of the facial actions of an expression. Possible implications for the field of affective computing and clinical research are discussed.

[1]  K. Scherer,et al.  Studying the dynamics of emotional expression using synthesized facial muscle movements. , 2000, Journal of personality and social psychology.

[2]  Shuichi Nishio,et al.  Temporal differences in eye and mouth movements classifying facial expressions of smiles , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[3]  A. Senju,et al.  Direct gaze captures visuospatial attention , 2005 .

[4]  Jeffrey S. Maxwell,et al.  Human Amygdala Responsivity to Masked Fearful Eye Whites , 2004, Science.

[5]  R. Kleck,et al.  Effects of direct and averted gaze on the perception of facially communicated emotion. , 2005, Emotion.

[6]  A. Manstead,et al.  Effects of Dynamic Aspects of Facial Expressions: A Review , 2013 .

[7]  J. Piven,et al.  Visual Scanning of Faces in Autism , 2002, Journal of autism and developmental disorders.

[8]  Garrison W. Cottrell,et al.  Transmitting and decoding facial expressions of emotion , 2004 .

[9]  Dominic J. Packer,et al.  On Being Both With Us and Against Us: A Normative Conflict Model of Dissent in Social Groups , 2008, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[10]  L. Miles,et al.  A critical review of the eyes test as a measure of social‐cognitive impairment , 2008 .

[11]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[12]  I. Pilowsky,et al.  Towards the quantification of facial expressions with the use of a mathematic model of the face , 1986 .

[13]  Kostas Karpouzis,et al.  Towards modeling embodied conversational agent character profiles using appraisal theory predictions in expression synthesis , 2009, Applied Intelligence.

[14]  Craig A. Smith,et al.  The psychology of facial expression: A Componential Approach to the meaning of facial expressions , 1997 .

[15]  Christine L. Lisetti,et al.  Adapting Psychologically Grounded Facial Emotional Expressions to Different Anthropomorphic Embodiment Platforms , 2007, FLAIRS.

[16]  W. Sato,et al.  Dynamic fearful expressions enhance gaze-triggered attention orienting in high and low anxiety individuals , 2009 .

[17]  U. Hess,et al.  The timing of appraisals. , 2007, Emotion.

[18]  D. Perrett,et al.  Being the target of another’s emotion: a PET study , 2003, Neuropsychologia.

[19]  K. Scherer,et al.  Sequential unfolding of novelty and pleasantness appraisals of odors: evidence from facial electromyography and autonomic reactions. , 2009, Emotion.

[20]  F. Amthor,et al.  Avatar Assistant: Improving Social Skills in Students with an ASD Through a Computer-Based Intervention , 2011, Journal of autism and developmental disorders.

[21]  M. Bindemann,et al.  How do eye gaze and facial expression interact? , 2008 .

[22]  Trudi Schoop,et al.  Motion and Emotion , 2000 .

[23]  S. Baron-Cohen,et al.  The "Reading the Mind in the Eyes" Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. , 2001, Journal of child psychology and psychiatry, and allied disciplines.

[24]  Darío Páez,et al.  The ‘Black Sheep Effect’: Social Categorization, Rejection of Ingroup Deviates, and Perception of Group Variability , 1994 .

[25]  A. Todorov,et al.  The functional basis of face evaluation , 2008, Proceedings of the National Academy of Sciences.

[26]  Oliver G. B. Garrod,et al.  Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time , 2014, Current Biology.

[27]  K. Scherer,et al.  Interaction effects of perceived gaze direction and dynamic facial expression: Evidence for appraisal theories of emotion , 2007 .

[28]  Radoslaw Niewiadomski,et al.  Evaluation of multimodal sequential expressions of emotions in ECA , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[29]  R. J. Dolan,et al.  Human Amygdala Responses to Fearful Eyes , 2002, NeuroImage.

[30]  Dominic Abrams,et al.  Membership status and subjective group dynamics: who triggers the black sheep effect? , 2010, Journal of personality and social psychology.

[31]  K. Scherer,et al.  Affect bursts: dynamic patterns of facial expression. , 2011, Emotion.

[32]  P. Schyns,et al.  A mechanism for impaired fear recognition after amygdala damage , 2005, Nature.

[33]  K. Scherer,et al.  First evidence for differential and sequential efferent effects of stimulus relevance and goal conduciveness appraisal , 2007, Biological Psychology.

[34]  S. Baron-Cohen,et al.  Reading the Eyes: Evidence for the Role of Perception in the Development of a Theory of Mind , 1992 .

[35]  P. Ekman.  An argument for basic emotions , 1992 .

[36]  Marianne Schmid Mast,et al.  Emotion Recognition: The Role of Featural and Configural Face Information , 2013, Quarterly journal of experimental psychology.

[37]  C. Deruelle,et al.  Motion and Emotion: A Novel Approach to the Study of Face Processing by Young Autistic Children , 2001, Journal of autism and developmental disorders.

[38]  Patrik Vuilleumier,et al.  Cognitive science: Staring fear in the face , 2005, Nature.

[39]  Radoslaw Niewiadomski,et al.  Modeling Emotional Expressions as Sequences of Behaviors , 2009, IVA.

[40]  Patrik Vuilleumier,et al.  Self-relevance processing in the human amygdala: gaze direction, facial expression, and emotion intensity. , 2009, Emotion.

[41]  K. Scherer,et al.  Appraisal processes in emotion. , 2003 .

[42]  K. Scherer,et al.  FACSGen 2.0 animation software: Generating 3D FACS-valid facial expressions for emotion research , 2012 .

[43]  K. Scherer.  The dynamic architecture of emotion: Evidence for the component process model , 2009 .

[44]  K. LaBar,et al.  Garner interference reveals dependencies between emotional expression and gaze in face perception. , 2007, Emotion.

[45]  D. Moore.  Reassessing emotion recognition performance in people with mental retardation: a review. , 2001, American journal of mental retardation : AJMR.

[46]  P. Ekman.  The argument and evidence about universals in facial expressions of emotion. , 1989 .

[47]  A. Karmiloff-Smith,et al.  Are children with autism blind to the mentalistic significance of the eyes , 1995 .

[48]  A. Calder,et al.  Neural mechanisms of social attention , 2009, Trends in Cognitive Sciences.

[49]  Georgina Randsley de Moura,et al.  Bribery, blackmail, and the double standard for leader transgressions. , 2013 .

[50]  D. Abrams,et al.  A double standard when group members behave badly: transgression credit to ingroup leaders. , 2013, Journal of personality and social psychology.

[51]  Herschel W. Leibowitz,et al.  Do the Eyes Have It , 1976 .

[52]  Emma Ashwin,et al.  Can emotion recognition be taught to children with autism spectrum conditions? , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[53]  V. Bruce,et al.  Do the eyes have it? Cues to the direction of social attention , 2000, Trends in Cognitive Sciences.

[54]  Dominic Abrams,et al.  Social identity and intragroup differentiation as subjective social control , 1998 .

[55]  K. Scherer,et al.  Are facial expressions of emotion produced by categorical affect programs or dynamically driven by appraisal? , 2007, Emotion.

[56]  G. Alpers,et al.  Happy mouth and sad eyes: scanning emotional facial expressions. , 2011, Emotion.

[57]  Garrison W. Cottrell,et al.  Transmitting and Decoding Facial Expressions , 2005, Psychological science.

[58]  P. Viviani,et al.  The Identification of Unfolding Facial Expressions , 2012, Perception.

[59]  Georgina Randsley de Moura,et al.  The Effects of Cost, Normative Support, and Issue Importance on Motivation to Persuade In-Group Deviants , 2010 .

[60]  Susanne Kaiser,et al.  Sequential Patterning of Facial Actions in the Production and Perception of Emotional Expressions , 2011 .

[61]  K. Scherer.  Appraisal considered as a process of multilevel sequential checking. , 2001 .

[62]  Eva G Krumhuber,et al.  FACSGen 2.0 animation software: generating three-dimensional FACS-valid facial expressions for emotion research. , 2012, Emotion.

[63]  Klaus R. Scherer,et al.  On the Sequential Nature of Appraisal Processes: Indirect Evidence from a Recognition Task , 1999 .

[64]  S. Baron-Cohen,et al.  Another advanced test of theory of mind: evidence from very high functioning adults with autism or asperger syndrome. , 1997, Journal of child psychology and psychiatry, and allied disciplines.

[65]  D. Abrams,et al.  Being better by being right: subjective group dynamics and derogation of in-group deviants when generic norms are undermined. , 2001, Journal of personality and social psychology.

[66]  D. Abrams,et al.  Innovation credit: when can leaders oppose their group's norms? , 2008, Journal of personality and social psychology.

[67]  R. Kleck,et al.  Perceived Gaze Direction and the Processing of Facial Displays of Emotion , 2003, Psychological science.

[68]  P. Ekman.  Darwin, Deception, and Facial Expression , 2003, Annals of the New York Academy of Sciences.

[69]  J. Hietanen,et al.  Detection of emotional faces is modulated by the direction of eye gaze. , 2011, Emotion.

[70]  Danielle Ropar,et al.  Do the eyes have it? Inferring mental states from animated faces in autism. , 2007, Child development.

[71]  M. Mermillod,et al.  Is eye contact the key to the social brain? , 2010, Behavioral and Brain Sciences.

[72]  R. Hobson.  The autistic child's appraisal of expressions of emotion. , 1986, Journal of child psychology and psychiatry, and allied disciplines.

[73]  R. Joseph,et al.  Holistic and part-based face recognition in children with autism. , 2003, Journal of child psychology and psychiatry, and allied disciplines.

[74]  Andrés Fernández-Martín,et al.  Can the eyes reveal a person’s emotions? Biasing role of the mouth expression , 2013 .