Task-dependent enhancement of facial expression and identity representations in human cortex

&NA; What cortical mechanisms allow humans to easily discern the expression or identity of a face? Subjects detected changes in expression or identity of a stream of dynamic faces while we measured BOLD responses from topographically and functionally defined areas throughout the visual hierarchy. Responses in dorsal areas increased during the expression task, whereas responses in ventral areas increased during the identity task, consistent with previous studies. Similar to ventral areas, early visual areas showed increased activity during the identity task. If visual responses are weighted by perceptual mechanisms according to their magnitude, these increased responses would lead to improved attentional selection of the task‐appropriate facial aspect. Alternatively, increased responses could be a signature of a sensitivity enhancement mechanism that improves representations of the attended facial aspect. Consistent with the latter sensitivity enhancement mechanism, attending to expression led to enhanced decoding of exemplars of expression both in early visual and dorsal areas relative to attending identity. Similarly, decoding identity exemplars when attending to identity was improved in dorsal and ventral areas. We conclude that attending to expression or identity of dynamic faces is associated with increased selectivity in representations consistent with sensitivity enhancement.

[1]  Alex R. Wade,et al.  Visual field maps and stimulus selectivity in human ventral occipital cortex , 2005, Nature Neuroscience.

[2]  Justin L Gardner,et al.  Encoding of graded changes in spatial specificity of prior cues in human visual cortex. , 2014, Journal of neurophysiology.

[3]  S. Treue,et al.  Feature-Based Attention Increases the Selectivity of Population Responses in Primate Visual Cortex , 2004, Current Biology.

[4]  John H. R. Maunsell,et al.  Feature-based attention in visual cortex , 2006, Trends in Neurosciences.

[5]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[6]  R. Desimone,et al.  Attention Increases Sensitivity of V4 Neurons , 2000, Neuron.

[7]  K. Grill-Spector,et al.  The evolution of face processing networks , 2015, Trends in Cognitive Sciences.

[8]  H. Bülthoff,et al.  What the Human Brain Likes About Facial Motion , 2012, Cerebral cortex.

[9]  J. Maunsell,et al.  Using Neuronal Populations to Study the Mechanisms Underlying Spatial and Feature Attention , 2011, Neuron.

[10]  K. Lander,et al.  Recognizing Face Identity from Natural and Morphed Smiles , 2006, Quarterly journal of experimental psychology.

[11]  Tzvi Ganel,et al.  Effects of familiarity on the perceptual integrality of the identity and expression of faces: the parallel-route hypothesis revisited. , 2004, Journal of experimental psychology. Human perception and performance.

[12]  R. M. Siegel,et al.  Maps of Visual Space in Human Occipital Cortex Are Retinotopic, Not Spatiotopic , 2008, The Journal of Neuroscience.

[13]  Ingrid R. Olson,et al.  More Than Meets the Eye: The Merging of Perceptual and Conceptual Knowledge in the Anterior Temporal Face Area , 2016, Front. Hum. Neurosci..

[14]  Mark D'Esposito,et al.  Attention Selectively Modifies the Representation of Individual Faces in the Human Brain , 2013, The Journal of Neuroscience.

[15]  K. Grill-Spector,et al.  fMR-adaptation: a tool for studying the functional properties of human cortical neurons. , 2001, Acta psychologica.

[16]  Karl J. Friston,et al.  Network Interactions Explain Sensitivity to Dynamic Faces in the Superior Temporal Sulcus , 2014, Cerebral cortex.

[17]  Stephen A. Engel,et al.  Linear systems analysis of the fMRI signal , 2012, NeuroImage.

[18]  Umberto Castiello,et al.  The human temporal lobe integrates facial form and motion: evidence from fMRI and ERP studies , 2003, NeuroImage.

[19]  J. Haxby,et al.  Distinct representations of eye gaze and identity in the distributed human neural system for face perception , 2000, Nature Neuroscience.

[20]  K. Lander,et al.  Independence of face identity and expression processing: exploring the role of motion , 2015, Front. Psychol..

[21]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[22]  A. J. Mistlin,et al.  Specialized face processing and hemispheric asymmetry in man and monkey: Evidence from single unit and reaction time studies , 1988, Behavioural Brain Research.

[23]  B. de Gelder,et al.  The Neural Basis of Individual Face and Object Perception , 2016, Front. Hum. Neurosci..

[24]  M. Carrasco Visual attention: The past 25 years , 2011, Vision Research.

[25]  D J Heeger,et al.  Robust multiresolution alignment of MRI brain volumes , 2000, Magnetic resonance in medicine.

[26]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[27]  N. Kanwisher,et al.  Feedback of pVisual Object Information to Foveal Retinotopic Cortex , 2008, Nature Neuroscience.

[28]  L. Jakobson,et al.  A sex difference in interference between identity and expression judgments with static but not dynamic faces. , 2013, Journal of vision.

[29]  Scott D. Slotnick,et al.  The Fusiform Face Area , 2013 .

[30]  Isabelle Bülthoff,et al.  Identity information content depends on the type of facial movement , 2016, Scientific Reports.

[31]  Galit Yovel,et al.  Two neural pathways of face processing: A critical evaluation of current models , 2015, Neuroscience & Biobehavioral Reviews.

[32]  Philippe G Schyns,et al.  Decoding face categories in diagnostic subregions of primary visual cortex , 2013, The European journal of neuroscience.

[33]  Leslie G. Ungerleider,et al.  Neural processing of emotional faces requires attention , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Bruno Rossion,et al.  Early lateralization and orientation tuning for face, word, and object processing in the visual cortex , 2003, NeuroImage.

[35]  N. Kriegeskorte,et al.  Faciotopy—A face-feature map with face-like topology in the human occipital face area , 2015, Cortex.

[36]  Leslie G. Ungerleider,et al.  Repetition suppression of faces is modulated by emotion. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[37]  N. Kanwisher,et al.  The fusiform face area: a cortical region specialized for the perception of faces , 2006, Philosophical Transactions of the Royal Society B: Biological Sciences.

[38]  Martin A. Giese,et al.  Semantic 3D motion retargeting for facial animation , 2006, APGV '06.

[39]  Jeesun Kim,et al.  Emotional expressions evoke a differential response in the fusiform face area , 2013, Front. Hum. Neurosci..

[40]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[41]  Martin Breidt,et al.  Quantifying human sensitivity to spatio-temporal information in dynamic faces , 2013, Vision Research.

[42]  Kathleen Lennon,et al.  Sex Difference , 2021, Encyclopedia of the UN Sustainable Development Goals.

[43]  Rebecca Saxe,et al.  A Common Neural Code for Perceived and Inferred Emotion , 2014, The Journal of Neuroscience.

[44]  Alexander Todorov,et al.  Brain systems for assessing the affective value of faces , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[45]  Scott O Murray,et al.  The effects of spatial attention in early human visual cortex are stimulus independent. , 2008, Journal of vision.

[46]  Swann Pichon,et al.  Similarities and differences in perceiving threat from dynamic faces and bodies. An fMRI study , 2011, NeuroImage.

[47]  A. O'Toole,et al.  Recognizing moving faces: a psychological and neural synthesis , 2002, Trends in Cognitive Sciences.

[48]  G. McCarthy,et al.  Dynamic perception of facial affect and identity in the human brain. , 2003, Cerebral Cortex.

[49]  Giuseppe Iaria,et al.  The correlates of subjective perception of identity and expression in the face network: An fMRI adaptation study , 2009, NeuroImage.

[50]  F. Tong,et al.  Neural mechanisms of object-based attention. , 2015, Cerebral cortex.

[51]  B. Efron Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods , 1981 .

[52]  J. Sergent,et al.  Segregated processing of facial identity and emotion in the human brain: A pet study , 1994 .

[53]  Leslie G. Ungerleider,et al.  Multiple visual areas in the caudal superior temporal sulcus of the macaque , 1986, The Journal of comparative neurology.

[54]  Adrian T. Lee,et al.  fMRI of human visual cortex , 1994, Nature.

[55]  S. Dehaene,et al.  The priming method: imaging unconscious repetition priming reveals an abstract representation of number in the parietal lobes. , 2001, Cerebral cortex.

[56]  Zhe Wang,et al.  On the facilitative effects of face motion on face recognition and its development , 2014, Front. Psychol..

[57]  B. Wandell,et al.  Visual Field Maps in Human Cortex , 2007, Neuron.

[58]  T. Poggio,et al.  Cognitive neuroscience: Neural mechanisms for the recognition of biological movements , 2003, Nature Reviews Neuroscience.

[59]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[60]  M. Tarr,et al.  The Fusiform Face Area is Part of a Network that Processes Faces at the Individual Level , 2000, Journal of Cognitive Neuroscience.

[61]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[62]  X Hu,et al.  Retrospective estimation and correction of physiological fluctuation in functional MRI , 1995, Magnetic resonance in medicine.

[63]  A. Young,et al.  Understanding the recognition of facial identity and facial expression , 2005, Nature Reviews Neuroscience.

[64]  R. Desimone,et al.  Attention Increases Sensitivity of V4 Neurons , 2000, Neuron.

[65]  Daniel Algom,et al.  Half a century of research on Garner interference and the separability-integrality distinction. , 2016, Psychological bulletin.

[66]  Nikolaus Kriegeskorte,et al.  Comparison of multivariate classifiers and response normalizations for pattern-information fMRI , 2010, NeuroImage.

[67]  I. Biederman,et al.  Loci of the release from fMRI adaptation for changes in facial expression, identity, and viewpoint. , 2010, Journal of vision.

[68]  Justin L. Gardner,et al.  Feature-Specific Attentional Priority Signals in Human Cortex , 2011, The Journal of Neuroscience.

[69]  Justin L. Gardner,et al.  Cortical Correlates of Human Motion Perception Biases , 2014, The Journal of Neuroscience.

[70]  D. Heeger,et al.  Attentional Enhancement via Selection and Pooling of Early Sensory Responses in Human Visual Cortex , 2011, Neuron.

[71]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[72]  Timothy J. Andrews,et al.  Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe , 2004, NeuroImage.

[73]  Marlene Behrmann,et al.  Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis , 2011, Proceedings of the National Academy of Sciences.

[74]  Keiji Tanaka,et al.  Human Ocular Dominance Columns as Revealed by High-Field Functional Magnetic Resonance Imaging , 2001, Neuron.

[75]  W. R. Garner,et al.  Interaction of stimulus dimensions in concept and choice processes , 1976, Cognitive Psychology.

[76]  Jeffrey R. Binder,et al.  Some neurophysiological constraints on models of word naming , 2005, NeuroImage.

[77]  S. Schweinberger,et al.  Asymmetric relationships among perceptions of facial identity, emotion, and facial speech. , 1998, Journal of experimental psychology. Human perception and performance.

[78]  T. Allison,et al.  Face-Specific Processing in the Human Fusiform Gyrus , 1997, Journal of Cognitive Neuroscience.

[79]  J. Schultz,et al.  Natural facial motion enhances cortical responses to faces , 2009, Experimental Brain Research.

[80]  Hui Zhang,et al.  Face-selective regions differ in their ability to classify facial expressions , 2016, NeuroImage.

[81]  John H. R. Maunsell,et al.  Attention to both space and feature modulates neuronal responses in macaque area V4. , 2000, Journal of neurophysiology.

[82]  G. Yovel,et al.  Successful Decoding of Famous Faces in the Fusiform Face Area , 2015, PloS one.

[83]  T. Braver,et al.  BOLD Correlates of Trial-by-Trial Reaction Time Variability in Gray and White Matter: A Multi-Study fMRI Analysis , 2009, PloS one.

[84]  Giedrius T Buracas,et al.  The Effect of Spatial Attention on Contrast Response Functions in Human Visual Cortex , 2007, The Journal of Neuroscience.

[85]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[86]  R. Desimone,et al.  Neural Mechanisms of Object-Based Attention , 2014, Science.

[87]  N. Kanwisher Functional specificity in the human brain: A window into the functional architecture of the mind , 2010, Proceedings of the National Academy of Sciences.

[88]  R. Dolan,et al.  fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. , 2004, Journal of neurophysiology.

[89]  D. Perrett,et al.  Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. , 1996, Journal of neurophysiology.

[90]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[91]  Barbara A Dosher,et al.  Blood oxygenation level-dependent contrast response functions identify mechanisms of covert attention in early visual areas , 2008, Proceedings of the National Academy of Sciences.

[92]  Alexander G. Huth,et al.  Attention During Natural Vision Warps Semantic Representation Across the Human Brain , 2013, Nature Neuroscience.

[93]  Christopher J. Fox,et al.  It doesn't matter how you feel. The facial identity aftereffect is invariant to changes in facial expression. , 2008, Journal of vision.

[94]  B. Wandell Computational neuroimaging of human visual cortex. , 1999, Annual review of neuroscience.

[95]  A. Dickson On Evolution , 1884, Science.

[96]  Li Fei-Fei,et al.  Neural mechanisms of rapid natural scene categorization in human visual cortex , 2009, Nature.

[97]  C. Koch,et al.  Attention activates winner-take-all competition among visual filters , 1999, Nature Neuroscience.

[98]  Leslie G. Ungerleider,et al.  Distributed representation of objects in the human ventral visual pathway. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[99]  Matthew F. Peterson,et al.  Statistical decision theory to relate neurons to behavior in the study of covert visual attention , 2009, Vision Research.

[100]  B. C. Motter,et al.  Neural correlates of feature selective memory and pop-out in extrastriate area V4 , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[101]  Kenneth F. Valyear,et al.  The involvement of the “fusiform face area” in processing facial expression , 2005, Neuropsychologia.

[102]  Gaurav Singal,et al.  Lateralization of face processing in the human brain , 2012, Proceedings of the Royal Society B: Biological Sciences.

[103]  Reliability and reproducibility of brain tissue segmentation and volumetry of MR scans , 2001, NeuroImage.

[104]  Winrich A. Freiwald,et al.  Contrasting Specializations for Facial Motion within the Macaque Face-Processing System , 2015, Current Biology.

[105]  J. Bisley The neural basis of visual attention , 2011, The Journal of physiology.

[106]  R. J Dolan,et al.  Common and distinct neural responses during direct and incidental processing of multiple facial emotions , 2003, NeuroImage.

[107]  S. Kastner,et al.  Attention in the real world: toward understanding its neural basis , 2014, Trends in Cognitive Sciences.

[108]  Martin Eimer,et al.  Facial identity and facial expression are initially integrated at visual perceptual stages of face processing , 2016, Neuropsychologia.

[109]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[110]  Henning U. Voss,et al.  High-field functional magnetic resonance imaging of vocalization processing in marmosets , 2014, Scientific Reports.

[111]  Michael J. Brammer,et al.  A preferential increase in the extrastriate response to signals of danger , 2003, NeuroImage.

[112]  H. Bülthoff,et al.  The use of facial motion and facial form during the processing of identity , 2003, Vision Research.

[113]  Naiqi G Xiao,et al.  Elastic facial movement influences part-based but not holistic processing. , 2013, Journal of experimental psychology. Human perception and performance.

[114]  A M Burton,et al.  Asymmetric dependencies in perceiving identity and emotion: Experiments with morphed faces , 1999, Perception & psychophysics.

[115]  R. Dolan,et al.  Effects of Attention and Emotion on Face Processing in the Human Brain An Event-Related fMRI Study , 2001, Neuron.

[116]  Karl J. Friston,et al.  Top-Down Control of Visual Responses to Fear by the Amygdala , 2013, The Journal of Neuroscience.

[117]  Rainer Goebel,et al.  Horizontal tuning for faces originates in high-level Fusiform Face Area , 2016, Neuropsychologia.

[118]  H. Levitt Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[119]  Doris Y. Tsao,et al.  Comparing face patch systems in macaques and humans , 2008, Proceedings of the National Academy of Sciences.

[120]  J. Lange,et al.  A Model of Biological Motion Perception from Configural Form Cues , 2006, The Journal of Neuroscience.

[121]  Stefan Pollmann,et al.  Investigating the brain basis of facial expression perception using multi-voxel pattern analysis , 2015, Cortex.

[122]  N. Sadato,et al.  Attention to emotion modulates fMRI activity in human right superior temporal sulcus. , 2001, Brain research. Cognitive brain research.

[123]  Steen Moeller,et al.  T 1 weighted brain images at 7 Tesla unbiased for Proton Density, T 2 ⁎ contrast and RF coil receive B 1 sensitivity with simultaneous vessel visualization , 2009, NeuroImage.

[124]  Kathrin Cohen Kadosh,et al.  Task-dependent Activation of Face-sensitive Cortex: An fMRI Adaptation Study , 2010, Journal of Cognitive Neuroscience.

[125]  John H. R. Maunsell,et al.  Neuronal Mechanisms of Visual Attention. , 2015, Annual review of vision science.

[126]  D G Pelli,et al.  Uncertainty explains many aspects of visual contrast detection and discrimination. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[127]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[128]  Wei Ji Ma,et al.  Near-optimal integration of facial form and motion , 2017, Scientific Reports.

[129]  Leslie G. Ungerleider,et al.  Selective dissociation between core and extended regions of the face processing network in congenital prosopagnosia. , 2014, Cerebral cortex.

[130]  P. Verghese Visual Search and Attention A Signal Detection Theory Approach , 2001, Neuron.

[131]  Christopher J. Fox,et al.  Defining the face processing network: Optimization of the functional localizer in fMRI , 2009, Human brain mapping.

[132]  Johan D. Carlin,et al.  Decoding Face Exemplars from fMRI Responses: What Works, What Doesn't? , 2015, The Journal of Neuroscience.

[133]  Frank Tong,et al.  Spatial specificity of working memory representations in the early visual cortex. , 2014, Journal of vision.