Identification of Emotional Facial Expressions: Effects of Expression, Intensity, and Sex on Eye Gaze

The identification of emotional expressions is vital for social interaction, and can be affected by various factors, including the expressed emotion, the intensity of the expression, the sex of the face, and the gender of the observer. This study investigates how these factors affect the speed and accuracy of expression recognition, as well as dwell time on the two most significant areas of the face: the eyes and the mouth. Participants were asked to identify expressions from female and male faces displaying six expressions (anger, disgust, fear, happiness, sadness, and surprise), each with three levels of intensity (low, moderate, and normal). Overall, responses were fastest and most accurate for happy expressions, but slowest and least accurate for fearful expressions. More intense expressions were also classified most accurately. Reaction time showed a different pattern, with slowest response times recorded for expressions of moderate intensity. Overall, responses were slowest, but also most accurate, for female faces. Relative to male observers, women showed greater accuracy and speed when recognizing female expressions. Dwell time analyses revealed that attention to the eyes was about three times greater than on the mouth, with fearful eyes in particular attracting longer dwell times. The mouth region was attended to the most for fearful, angry, and disgusted expressions and least for surprise. These results extend upon previous findings to show important effects of expression, emotion intensity, and sex on expression recognition and gaze behaviour, and may have implications for understanding the ways in which emotion recognition abilities break down.

[1]  P. Ekman Are there basic emotions? , 1992, Psychological review.

[2]  Jeffrey S. Maxwell,et al.  Human Amygdala Responsivity to Masked Fearful Eye Whites , 2004, Science.

[3]  Joan Y. Chiao,et al.  Eye movements during emotion recognition in faces. , 2014, Journal of vision.

[4]  Zoltan Dienes,et al.  Using Bayes to get the most out of non-significant results , 2014, Front. Psychol..

[5]  Gina M. Grimshaw,et al.  A signal-detection analysis of sex differences in the perception of emotional faces , 2004, Brain and Cognition.

[6]  Judith A. Hall Gender Effects in Decoding Nonverbal Cues , 1978 .

[7]  David I. Perrett,et al.  Sex differences in the perception of affective facial expressions: Do men really lack emotional sensitivity? , 2005, Cognitive Processing.

[8]  Vicki Bruce,et al.  Evaluating the independence of sex and expression in judgments of faces , 2002, Perception & psychophysics.

[9]  Pia Rotshtein,et al.  Psychopathic traits are associated with reduced attention to the eyes of emotional faces among adult male non-offenders , 2015, Front. Hum. Neurosci..

[10]  D. Perrett,et al.  Reduced sensitivity to others’ fearful expressions in psychopathic individuals , 2004 .

[11]  Mitsuo Endo,et al.  Happy face advantage in recognizing facial expressions , 1995 .

[12]  Reginald B. Adams,et al.  The Intersection of Gender-Related Facial Appearance and Facial Displays of Emotion , 2015 .

[13]  M. T. Motley,et al.  Facial expression of emotion: A comparison of posed expressions versus spontaneous expressions in an interpersonal communication setting , 1988 .

[14]  R. J. Dolan,et al.  Human Amygdala Responses to Fearful Eyes , 2002, NeuroImage.

[15]  P. Greenfield,et al.  Infant sleeping arrangements and cultural values among contemporary Japanese mothers , 2014, Front. Psychol..

[16]  A. Guastella,et al.  Reduced eye gaze explains "fear blindness" in childhood psychopathic traits. , 2008, Journal of the American Academy of Child and Adolescent Psychiatry.

[17]  S. Shields,et al.  Gender and Emotion , 2006 .

[18]  R. Adolphs,et al.  Impaired Judgments of Sadness But Not Happiness Following Bilateral Amygdala Damage , 2004, Journal of Cognitive Neuroscience.

[19]  Steven L. Neuberg,et al.  The confounded nature of angry men and happy women. , 2007, Journal of personality and social psychology.

[20]  D. Skuse,et al.  Impaired sadness recognition is linked to social interaction deficit in autism , 2007, Neuropsychologia.

[21]  S. Hutton,et al.  Age differences in emotion recognition skills and the visual scanning of emotion faces. , 2007, The journals of gerontology. Series B, Psychological sciences and social sciences.

[22]  Philippe G Schyns,et al.  Smile Through Your Fear and Sadness , 2009, Psychological science.

[23]  E. Ashby Plant,et al.  The Influence of Gender and Social Role on the Interpretation of Facial Expressions , 2004 .

[24]  Kerry Hourigan,et al.  Wake transition of a rolling sphere , 2011, J. Vis..

[25]  Garrison W. Cottrell,et al.  Transmitting and Decoding Facial Expressions , 2005, Psychological science.

[26]  B. Parkinson,et al.  Personality and Social Psychology Review Do Facial Movements Express Emotions or Communicate Motives? Personality and Social Psychology Review Additional Services and Information for Do Facial Movements Express Emotions or Communicate Motives? Emotion Expression vs. Motive Communication Emotion Expr , 2022 .

[27]  S. Mineka,et al.  Fears, phobias, and preparedness: toward an evolved module of fear and fear learning. , 2001, Psychological review.

[28]  Luna C. Muñoz Callous-unemotional traits are related to combined deficits in recognizing afraid faces and body poses. , 2009, Journal of the American Academy of Child and Adolescent Psychiatry.

[29]  Karen L. Schmidt,et al.  Human facial expressions as adaptations: Evolutionary questions in facial expression research. , 2001, American journal of physical anthropology.

[30]  P. Vuilleumier,et al.  How brains beware: neural mechanisms of emotional attention , 2005, Trends in Cognitive Sciences.

[31]  H. Friedman,et al.  Sex differences in nonverbal expressiveness: Emotional expression, personality, and impressions , 1993 .

[32]  Stefan J. Kiebel,et al.  Amygdala damage affects event‐related potentials for fearful faces at specific time windows , 2009, Human brain mapping.

[33]  D. Matsumoto,et al.  Idiocentric and allocentric differences in emotional expression, experience, and the coherence between expression and experience , 2001 .

[34]  Krogman Wm,et al.  Bertram Shirley Kraus 1913-1970. A biographical sketch. , 1970 .

[35]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[36]  E. Gordon,et al.  Schizophrenia and affective disorder show different visual scanning behavior for faces: a trait versus state-based distinction? , 2002, Biological Psychiatry.

[37]  Roman Feiman,et al.  Expressing fear enhances sensory acquisition , 2008, Nature Neuroscience.

[38]  Arne Öhman,et al.  Fear and anxiety as emotional phenomena: Clinical phenomenology, evolutionary perspectives, and information-processing mechanisms. , 1993 .

[39]  G. Alpers,et al.  Categorization and evaluation of emotional faces in psychopathic women , 2008, Psychiatry Research.

[40]  M. Coltheart,et al.  Photographs of facial expression: Accuracy, response times, and ratings of intensity , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[41]  J. Douglas,et al.  Visual scanning in the recognition of facial affect: is there an observer sex difference? , 2009, Journal of vision.

[42]  Mirja Tenhunen,et al.  Faster Choice-Reaction Times to Positive than to Negative Facial Expressions: The Role of Cognitive , 2003 .

[43]  E. Fox,et al.  Facial Expressions of Emotion: Are Angry Faces Detected More Efficiently? , 2000, Cognition & emotion.

[44]  Jean-Marc Fellous,et al.  Facial resemblance to emotions: group differences, impression effects, and race stereotypes. , 2010, Journal of personality and social psychology.

[45]  D. Perrett,et al.  Categorical Perception of Morphed Facial Expressions , 1996 .

[46]  J. Piven,et al.  Visual Scanning of Faces in Autism , 2002, Journal of autism and developmental disorders.

[47]  Kun Guo,et al.  Holistic Gaze Strategy to Categorize Facial Expression of Varying Intensities , 2012, PloS one.

[48]  P. Schyns,et al.  A mechanism for impaired fear recognition after amygdala damage , 2005, Nature.

[49]  P. Ekman Universals and cultural differences in facial expressions of emotion. , 1972 .

[50]  A. Young,et al.  Configural information in facial expression perception. , 2000, Journal of experimental psychology. Human perception and performance.

[51]  U. Hess,et al.  The Intensity of Emotional Facial Expressions and Decoding Accuracy , 1997 .

[52]  Ioannis Pitas,et al.  An analysis of facial expression recognition under partial facial image occlusion , 2008, Image Vis. Comput..

[53]  S. Rauch,et al.  Masked Presentations of Emotional Facial Expressions Modulate Amygdala Activity without Explicit Knowledge , 1998, The Journal of Neuroscience.

[54]  F. Doré,et al.  Accuracy and Latency of Judgment of Facial Expressions of Emotions , 1983, Perceptual and motor skills.

[55]  Paul Ekman,et al.  Facial Expressions of Emotion: New Findings, New Questions , 1992 .

[56]  P. Ekman Facial expression and emotion. , 1993, The American psychologist.

[57]  Judith A. Hall,et al.  Gender differences in nonverbal communication of emotion , 2000 .

[58]  Ralf Engbert,et al.  Microsaccades uncover the orientation of covert attention , 2003, Vision Research.

[59]  Naomi G. Rotter,et al.  Sex differences in the encoding and decoding of negative facial emotions , 1988 .

[60]  S. Baron-Cohen The extreme male brain theory of autism , 2002, Trends in Cognitive Sciences.

[61]  C. Frith Role of facial expressions in social interactions , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[62]  S. Rauch,et al.  A functional MRI study of human amygdala responses to facial expressions of fear versus anger. , 2001, Emotion.

[63]  A. Öhman,et al.  Preparedness and preattentive associative learning: Electrodermal conditioning to masked stimuli. , 1995 .

[64]  Leslie G. Ungerleider,et al.  Neural processing of emotional faces requires attention , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[65]  M. Morgan,et al.  Sex differences in scanning faces: Does attention to the eyes explain female superiority in facial expression recognition? , 2010 .

[66]  A. J. Fridlund,et al.  Facial Expressions , 2018, Encyclopedia of Evolutionary Psychological Science.

[67]  G. Alpers,et al.  Happy mouth and sad eyes: scanning emotional facial expressions. , 2011, Emotion.

[68]  J. Tanaka,et al.  The NimStim set of facial expressions: Judgments from untrained research participants , 2009, Psychiatry Research.

[69]  Lucy I Mullin,et al.  A female advantage in the recognition of emotional facial expressions: test of an evolutionary hypothesis , 2006 .

[70]  R. Riggio,et al.  The role of social skills in encoding posed and spontaneous facial expressions , 1988 .

[71]  Judith A. Hall,et al.  Gender differences in judgments of multiple emotions from facial expressions. , 2004, Emotion.

[72]  Ross A. Thompson,et al.  Sex differences in the recognition of infant facial expressions of emotion: The primary caretaker hypothesis , 1985 .

[73]  R. Dolan,et al.  Effects of Attention and Emotion on Face Processing in the Human Brain An Event-Related fMRI Study , 2001, Neuron.

[74]  Hillary Anger Elfenbein,et al.  On the universality and cultural specificity of emotion recognition: a meta-analysis. , 2002, Psychological bulletin.

[75]  P. Ekman,et al.  Unmasking the face : a guide to recognizing emotions from facial clues , 1975 .

[76]  A. Mizuno,et al.  A change of the leading player in flow Visualization technique , 2006, J. Vis..

[77]  Steven M. Gillespie,et al.  Relations of Distinct Psychopathic Personality Traits with Anxiety and Fear: Findings from Offenders and Non-Offenders , 2015, PloS one.

[78]  R. Dolan,et al.  Conscious and unconscious emotional learning in the human amygdala , 1998, Nature.

[79]  Abigail A. Marsh,et al.  Deficits in facial affect recognition among antisocial populations: A meta-analysis , 2008, Neuroscience & Biobehavioral Reviews.

[80]  G. Wilson,et al.  Sex, sexual orientation, and identification of positive and negative facial affect , 2004, Brain and Cognition.

[81]  Chris Rorden,et al.  Exogenous Orienting of Attention Depends upon the Ability to Execute Eye Movements , 2004, Current Biology.

[82]  Thierry d'Amato,et al.  Effects of emotion and identity on facial affect processing in schizophrenia , 2005, Psychiatry Research.

[83]  Stefanie Rukavina,et al.  Expression intensity, gender and facial emotion recognition: Women recognize only subtle facial emotions better than men. , 2010, Acta psychologica.