The Role of Emotional Content and Perceptual Saliency During the Programming of Saccades Toward Faces

Previous studies have shown that the human visual system can detect a face and elicit a saccadic eye movement toward it very efficiently compared to other categories of visual stimuli. In the first experiment, we tested the influence of facial expressions on fast face detection using a saccadic choice task. Face-vehicle pairs were simultaneously presented and participants were asked to saccade toward the target (the face or the vehicle). We observed that saccades toward faces were initiated faster, and more often in the correct direction, than saccades toward vehicles, regardless of the facial expressions (happy, fearful, or neutral). We also observed that saccade endpoints on face images were lower when the face was happy and higher when it was neutral. In the second experiment, we explicitly tested the detection of facial expressions. We used a saccadic choice task with emotional-neutral pairs of faces and participants were asked to saccade toward the emotional (happy or fearful) or the neutral face. Participants were faster when they were asked to saccade toward the emotional face. They also made fewer errors, especially when the emotional face was happy. Using computational modeling, we showed that this happy face advantage can, at least partly, be explained by perceptual factors. Also, saccade endpoints were lower when the target was happy than when it was fearful. Overall, we suggest that there is no automatic prioritization of emotional faces, at least for saccades with short latencies, but that salient local face features can automatically attract attention.

[1]  A. Kingstone,et al.  Natural gaze signaling in a social context , 2014 .

[2]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[3]  Louisa Kulke,et al.  Neural Mechanisms of Overt Attention Shifts to Emotional Faces , 2019, Neuroscience.

[4]  A. Ohman,et al.  The face in the crowd revisited: a threat advantage with schematic stimuli. , 2001, Journal of personality and social psychology.

[5]  S. Stigchel,et al.  How global is the global effect? The spatial characteristics of saccade averaging , 2013, Vision Research.

[6]  S Van der Stigchel,et al.  There is no attentional global effect: Attentional shifts are independent of the saccade endpoint. , 2015, Journal of vision.

[7]  Raymond J. Dolan,et al.  Subcortical amygdala pathways enable rapid face processing , 2014, NeuroImage.

[8]  G. Rhodes,et al.  A comparative view of face perception. , 2010, Journal of comparative psychology.

[9]  Martin Wegrzyn,et al.  Mapping the emotional face. How individual face parts contribute to successful emotion recognition , 2017, PloS one.

[10]  Charles E. Davis,et al.  Zapping 500 faces in less than 100 seconds: Evidence for extremely fast and sustained continuous visual search , 2018, Scientific Reports.

[11]  L. Nummenmaa,et al.  Eye-movement assessment of the time course in facial expression recognition: Neurophysiological implications , 2009, Cognitive, affective & behavioral neuroscience.

[12]  R. Adolphs Cognitive neuroscience: Cognitive neuroscience of human social behaviour , 2003, Nature Reviews Neuroscience.

[13]  Andrew W Young,et al.  The eyebrow frown: a salient social signal. , 2002, Emotion.

[14]  B. Strange,et al.  A fast pathway for fear in human amygdala , 2016, Nature Neuroscience.

[15]  O. Lipp,et al.  No effect of inversion on attentional and affective processing of facial expressions. , 2009, Emotion.

[16]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[17]  J. Theeuwes,et al.  Oculomotor Guidance and Capture by Irrelevant Faces , 2012, PloS one.

[18]  E. Keller,et al.  Dependence on target configuration of express saccade-related activity in the primate superior colliculus. , 1998, Journal of neurophysiology.

[19]  S. van der Stigchel,et al.  The image features of emotional faces that predict the initial eye movement to a face , 2021, Scientific Reports.

[20]  D. Kerzel,et al.  Face processing is enhanced in the left and upper visual hemi-fields , 2017 .

[21]  Garrison W. Cottrell,et al.  Transmitting and Decoding Facial Expressions , 2005, Psychological science.

[22]  Nathalie Guyader,et al.  Modelling Spatio-Temporal Saliency to Predict Gaze Direction for Short Videos , 2009, International Journal of Computer Vision.

[23]  Derrick J. Parkhurst,et al.  Scene content selected by active vision. , 2003, Spatial vision.

[24]  Martial Mermillod,et al.  Coarse scales are sufficient for efficient categorization of emotional facial expressions: Evidence from neural computation , 2010, Neurocomputing.

[25]  B. Wood,et al.  The evolutionary history of the human face , 2019, Nature Ecology & Evolution.

[26]  Manon Mulckhuyse The influence of emotional stimuli on the oculomotor system: A review of the literature , 2018, Cognitive, Affective, & Behavioral Neuroscience.

[27]  Daniel Lundqvist,et al.  Looking for foes and friends: perceptual and emotional factors when finding a face in the crowd. , 2005, Emotion.

[28]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[29]  Herbert Hoijtink,et al.  A tutorial on testing hypotheses using the Bayes factor. , 2019, Psychological methods.

[30]  L. Nummenmaa,et al.  Detection of emotional faces: salient physical features guide effective visual search. , 2008, Journal of experimental psychology. General.

[31]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[32]  R. Dolan,et al.  Distinct spatial frequency sensitivities for processing faces and emotional expressions , 2003, Nature Neuroscience.

[33]  Timothy D. Sweeny,et al.  Detecting and categorizing fleeting emotions in faces. , 2013, Emotion.

[34]  N. Emery,et al.  The eyes have it: the neuroethology, function and evolution of social gaze , 2000, Neuroscience & Biobehavioral Reviews.

[35]  Christel Devue,et al.  Faces are special, but facial expressions aren’t: Insights from an oculomotor capture paradigm , 2017, Attention, perception & psychophysics.

[36]  Daniel Smilek,et al.  Visual search for faces with emotional expressions. , 2008, Psychological bulletin.

[37]  James W Bisley,et al.  The neural instantiation of a priority map. , 2019, Current opinion in psychology.

[38]  Christian Keysers,et al.  The BOLD signal in the amygdala does not differentiate between dynamic facial expressions. , 2007, Social cognitive and affective neuroscience.

[39]  Karl J. Friston,et al.  A neuromodulatory role for the human amygdala in processing emotional facial expressions. , 1998, Brain : a journal of neurology.

[40]  A. Coutrot,et al.  How saliency, faces, and sound influence gaze in dynamic social scenes. , 2014, Journal of vision.

[41]  Michael B. Lewis,et al.  Face Detection: Mapping Human Performance , 2003, Perception.

[42]  G. Underwood,et al.  Salience of the lambs: a test of the saliency map hypothesis with pictures of emotive objects. , 2012, Journal of vision.

[43]  Tom Foulsham,et al.  Gaze allocation in a dynamic situation: Effects of social status and speaking , 2010, Cognition.

[44]  Martial Mermillod,et al.  A Rapid Subcortical Amygdala Route for Faces Irrespective of Spatial Frequency and Emotion , 2016, The Journal of Neuroscience.

[45]  Nathalie Guyader,et al.  Do low spatial frequencies explain the extremely fast saccades towards human faces? , 2017, Vision Research.

[46]  Y. Rosseel,et al.  Bain: a program for Bayesian testing of order constrained hypotheses in structural equation models , 2019, Journal of Statistical Computation and Simulation.

[47]  Jarrett D. Barker,et al.  The efficiency of dynamic and static facial expression recognition. , 2013, Journal of vision.

[48]  Rebecca M. Todd,et al.  Affective Salience Can Reverse the Effects of Stimulus-Driven Salience on Eye Movements in Complex Scenes , 2012, Front. Psychology.

[49]  Marie L. Smith,et al.  Transmission of Facial Expressions of Emotion Co-Evolved with Their Efficient Decoding in the Brain: Behavioral and Brain Evidence , 2009, PloS one.

[50]  Nathalie Guyader,et al.  The importance of recurrent top-down synaptic connections for the anticipation of dynamic emotions , 2019, Neural Networks.

[51]  L. Damjanovic,et al.  The eyes are sufficient to produce a threat superiority effect. , 2006, Emotion.

[52]  Ana García-Gutiérrez,et al.  Effects of Prime Task on Affective Priming By Facial Expressions of Emotion , 2007, The Spanish Journal of Psychology.

[53]  Luis Carretié,et al.  Electrophysiological differences in the processing of affective information in words and pictures , 2009, Cognitive, affective & behavioral neuroscience.

[54]  H. Lilliefors On the Kolmogorov-Smirnov Test for Normality with Mean and Variance Unknown , 1967 .

[55]  G. Gendolla,et al.  Detecting emotional faces and features in a visual search paradigm: are faces special? , 2006, Emotion.

[56]  Joseph E LeDoux Emotion Circuits in the Brain , 2000 .

[57]  Mark H. Johnson Subcortical face processing , 2005, Nature Reviews Neuroscience.

[58]  C. Koch,et al.  Faces and text attract gaze independent of the task: Experimental data and computer model. , 2009, Journal of vision.

[59]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[60]  Muriel Boucart,et al.  Finding faces, animals, and vehicles in far peripheral vision. , 2016, Journal of vision.

[61]  Margot J. Taylor,et al.  Attention inhibition of early cortical activation to fearful faces , 2010, Brain Research.

[62]  Arne Öhman,et al.  The role of the amygdala in human fear: Automatic detection of threat , 2005, Psychoneuroendocrinology.

[63]  Michael Angstadt,et al.  Beyond threat: Amygdala reactivity across multiple expressions of facial affect , 2006, NeuroImage.

[64]  L. Itti,et al.  Quantifying center bias of observers in free viewing of dynamic natural scenes. , 2009, Journal of vision.

[65]  M. Milders,et al.  Processing emotional stimuli: Comparison of saccadic and manual choice-reaction times , 2009 .

[66]  J. Findlay Global visual processing for saccadic eye movements , 1982, Vision Research.

[67]  P. Viviani,et al.  Is there a dynamic advantage for facial expressions? , 2011, Journal of vision.

[68]  Jan Theeuwes,et al.  Goal-driven, stimulus-driven, and history-driven selection. , 2019, Current opinion in psychology.

[69]  Jeffrey S. Racine,et al.  RStudio: A Platform-Independent IDE for R and Sweave , 2012 .

[70]  D. Lundqvist,et al.  Facial expressions of emotion (KDEF): Identification under different display-duration conditions , 2008, Behavior research methods.

[71]  M. Bindemann Scene and screen center bias early eye movements in scene viewing , 2010, Vision Research.

[72]  M. Tamietto,et al.  Neural bases of the non-conscious perception of emotional signals , 2010, Nature Reviews Neuroscience.

[73]  Hiromitsu Kobayashi,et al.  Unique morphology of the human eye , 1997, Nature.

[74]  G. Cottrell,et al.  EMPATH: A Neural Network that Categorizes Facial Expressions , 2002, Journal of Cognitive Neuroscience.

[75]  N. Guyader,et al.  Face perception influences the programming of eye movements , 2019, Scientific Reports.

[76]  A. Kingstone,et al.  The effect of emotional faces on eye movements and attention , 2007 .

[77]  Simon J. Thorpe,et al.  Low-Level Cues and Ultra-Fast Face Detection , 2011, Front. Psychology.

[78]  Dimitri J. Bayle,et al.  Unconsciously Perceived Fear in Peripheral Vision Alerts the Limbic System: A MEG Study , 2009, PloS one.

[79]  Stefanie I. Becker,et al.  Of toothy grins and angry snarls--open mouth displays contribute to efficiency gains in search for emotional faces. , 2012, Journal of vision.

[80]  H. Ellis 2 – THE ROLE OF THE RIGHT HEMISPHERE IN FACE PERCEPTION , 1983 .

[81]  Reginald B. Adams,et al.  Face gender and emotion expression: are angry women more like men? , 2009, Journal of vision.

[82]  Lauri Nummenmaa,et al.  Perceptual and affective mechanisms in facial expression recognition: An integrative review , 2016, Cognition & emotion.

[83]  Evie McCrum-Gardner,et al.  Which is the correct statistical test to use? , 2008, The British journal of oral & maxillofacial surgery.

[84]  Sébastien M. Crouzet,et al.  Fast saccades toward faces: face detection in just 100 ms. , 2010, Journal of vision.

[85]  G. Alpers,et al.  Happy mouth and sad eyes: scanning emotional facial expressions. , 2011, Emotion.

[86]  Artem V Belopolsky,et al.  Common Priority Map for Selection History, Reward and Emotion in the Oculomotor System , 2015, Perception.

[87]  J. Tanaka,et al.  The NimStim set of facial expressions: Judgments from untrained research participants , 2009, Psychiatry Research.

[88]  H. Demaree,et al.  Brain lateralization of emotional processing: historical roots and a future incorporating "dominance". , 2005, Behavioral and cognitive neuroscience reviews.

[89]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[90]  Martial Mermillod,et al.  Neural computation as a tool to differentiate perceptual from emotional processes: The case of anger superiority effect , 2009, Cognition.

[91]  E. Naito,et al.  Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. , 2004, Brain research. Cognitive brain research.

[92]  M. G. Calvo,et al.  Visual search of emotional faces: The role of affective content and featural distinctiveness , 2009 .

[93]  A. Inuggi,et al.  Task Difficulty and Response Complexity Modulate Affective Priming by Emotional Facial Expressions , 2014, Quarterly journal of experimental psychology.

[94]  Jeannette A M Lorteije,et al.  Priority Maps Explain the Roles of Value, Attention, and Salience in Goal-Oriented Behavior , 2014, The Journal of Neuroscience.

[95]  John C. Dalrymple-Alford,et al.  Sensitivity to genuine versus posed emotion specified in facial displays , 2010 .

[96]  R. VanRullen,et al.  Faces in the cloud: Fourier power spectrum biases ultrarapid face detection. , 2008, Journal of vision.

[97]  Simon J. Thorpe,et al.  Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited , 2006, Vision Research.

[98]  N. Guyader,et al.  Isolated face features are sufficient to elicit ultra-rapid and involuntary orienting responses toward faces , 2021, Journal of vision.

[99]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[100]  M. Mermillod,et al.  Evidence of fast and automatic gender bias in affective priming , 2015 .

[101]  S. Schweinberger,et al.  Combined effects of inversion and feature removal on N170 responses elicited by faces and car fronts , 2013, Brain and Cognition.

[102]  Jillian H. Fecteau,et al.  Salience, relevance, and firing: a priority map for target selection , 2006, Trends in Cognitive Sciences.

[103]  L. Nummenmaa,et al.  Time course of discrimination between emotional facial expressions: The role of visual saliency , 2011, Vision Research.

[104]  Chad R. Mortensen,et al.  Journal of Experimental Psychology: General Single-and Multiple-target Visual Search Tasks , 2022 .

[105]  J. Hietanen,et al.  Positive facial expressions are recognized faster than negative facial expressions, but why? , 2004, Psychological research.

[106]  J. Schultz,et al.  Use and Usefulness of Dynamic Face Stimuli for Face Perception Studies—a Review of Behavioral Findings and Methodology , 2018, Front. Psychol..