The bottom-up and top-down processing of faces in the human occipitotemporal cortex

Although face processing has been studied extensively, the dynamics of how face-selective cortical areas are engaged remains unclear. Here, we uncovered the timing of activation in core face-selective regions using functional Magnetic Resonance Imaging and Magnetoencephalography in humans. Processing of normal faces started in the posterior occipital areas and then proceeded to anterior regions. This bottom-up processing sequence was also observed even when internal facial features were misarranged. However, processing of two-tone Mooney faces lacking explicit prototypical facial features engaged top-down projection from the right posterior fusiform face area to right occipital face area. Further, face-specific responses elicited by contextual cues alone emerged simultaneously in the right ventral face-selective regions, suggesting parallel contextual facilitation. Together, our findings chronicle the precise timing of bottom-up, top-down, as well as context-facilitated processing sequences in the occipital-temporal face network, highlighting the importance of the top-down operations especially when faced with incomplete or ambiguous input.

[1]  C. M. Mooney Age in the development of closure ability in children. , 1957, Canadian journal of psychology.

[2]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[3]  T. Allison,et al.  Temporal Cortex Activation in Humans Viewing Eye and Mouth Movements , 1998, The Journal of Neuroscience.

[4]  Jason M. Gold,et al.  The response of face-selective cortex with single face parts and part combinations , 2012, Neuropsychologia.

[5]  Martin Luessi,et al.  MEG and EEG data analysis with MNE-Python , 2013, Front. Neuroinform..

[6]  Gilles Pourtois,et al.  View-independent coding of face identity in frontal and temporal cortices is modulated by familiarity: an event-related fMRI study , 2005, NeuroImage.

[7]  Jay Hegdé,et al.  Identifying faces in two-tone ('Mooney') images: A psychophysical and fMRI study , 2010 .

[8]  Andrew D. Engell,et al.  Facial expression and gaze-direction in human superior temporal sulcus , 2007, Neuropsychologia.

[9]  Doris Y. Tsao,et al.  A Cortical Region Consisting Entirely of Face-Selective Cells , 2006, Science.

[10]  G. Yovel,et al.  TMS Evidence for the Involvement of the Right Occipital Face Area in Early Face Processing , 2007, Current Biology.

[11]  Leslie G. Ungerleider,et al.  The Effect of Face Inversion on Activity in Human Neural Systems for Face and Object Perception , 1999, Neuron.

[12]  Zhe Wang,et al.  The other face of the other-race effect: An fMRI investigation of the other-race face categorization advantage , 2011, Neuropsychologia.

[13]  John A. Pyles,et al.  Dynamic Encoding of Face Information in the Human Fusiform Gyrus , 2014, Nature Communications.

[14]  Rankin W. McGugin,et al.  The Role of Experience in the Face-Selective Response in Right FFA , 2018, Cerebral cortex.

[15]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[16]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[17]  Á. Pascual-Leone,et al.  Repetitive TMS over posterior STS disrupts perception of biological motion , 2005, Vision Research.

[18]  Carlos F. Benitez-Quiroz,et al.  Facial color is an efficient mechanism to visually transmit emotion , 2018, Proceedings of the National Academy of Sciences.

[19]  N. Kanwisher,et al.  Stages of processing in face perception: an MEG study , 2002, Nature Neuroscience.

[20]  M. Tarr,et al.  The Fusiform Face Area is Part of a Network that Processes Faces at the Individual Level , 2000, Journal of Cognitive Neuroscience.

[21]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[22]  Doris Y. Tsao,et al.  Functional Compartmentalization and Viewpoint Generalization Within the Macaque Face-Processing System , 2010, Science.

[23]  Galit Yovel,et al.  A Revised Neural Framework for Face Processing. , 2015, Annual review of vision science.

[24]  Doris Y. Tsao,et al.  What Makes a Cell Face Selective? The Importance of Contrast , 2012, Neuron.

[25]  D. Mumford,et al.  Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency , 2002, Nature Neuroscience.

[26]  C. Jacques,et al.  The N170 : understanding the time-course of face perception in the human brain , 2011 .

[27]  Bruno Rossion,et al.  Fast periodic presentation of natural images reveals a robust face-selective electrophysiological response in the human brain. , 2015, Journal of vision.

[28]  A. Ishai,et al.  Effective connectivity within the distributed cortical network for face perception. , 2007, Cerebral cortex.

[29]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[30]  G. Yovel,et al.  The roles of perceptual and conceptual information in face recognition. , 2016, Journal of experimental psychology. General.

[31]  C. Granger Investigating Causal Relations by Econometric Models and Cross-Spectral Methods , 1969 .

[32]  Julie D. Golomb,et al.  A Neural Basis of Facial Action Recognition in Humans , 2016, The Journal of Neuroscience.

[33]  S. Tobimatsu,et al.  Neuromagnetic evidence that the right fusiform face area is essential for human face awareness: An intermittent binocular rivalry study , 2016, Neuroscience Research.

[34]  関原 謙介,et al.  Adaptive Spatial Filters for Electromagnetic Brain Imaging , 2008 .

[35]  Gabriel Curio,et al.  MEG/EEG sources of the 170-ms response to faces are co-localized in the fusiform gyrus , 2007, NeuroImage.

[36]  Elana Zion-Golumbic,et al.  Electrophysiological neural mechanisms for detection, configural analysis and recognition of faces , 2007, NeuroImage.

[37]  Scott O. Murray,et al.  Perceptual grouping and the interactions between visual cortical areas , 2004, Neural Networks.

[38]  Doris Y. Tsao,et al.  Faces and objects in macaque cerebral cortex , 2003, Nature Neuroscience.

[39]  Bruno Rossion,et al.  Figures and figure supplements , 2014 .

[40]  Doris Y. Tsao,et al.  The effect of face patch microstimulation on perception of faces and objects , 2017, Nature Neuroscience.

[41]  Katrin Amunts,et al.  The Cytoarchitecture of Domain-specific Regions in Human High-level Visual Cortex , 2016, Cerebral cortex.

[42]  Patrick Cavanagh,et al.  What's up in top-down processing? , 1991 .

[43]  Richard J. Harris,et al.  Neural Responses to Expression and Gaze in the Posterior Superior Temporal Sulcus Interact with Facial Identity , 2012, Cerebral cortex.

[44]  N. Kanwisher,et al.  The fusiform face area subserves face perception, not generic within-category identification , 2004, Nature Neuroscience.

[45]  Vinh Thai Nguyen,et al.  The superior temporal sulcus and the N170 during face processing: Single trial analysis of concurrent EEG–fMRI , 2014, NeuroImage.

[46]  R. Desimone,et al.  Neural Mechanisms of Object-Based Attention , 2014, Science.

[47]  A. O'Toole,et al.  Recognizing moving faces: a psychological and neural synthesis , 2002, Trends in Cognitive Sciences.

[48]  Dimitrios Pantazis,et al.  Similarity-based fusion of MEG and fMRI reveals spatio-temporal dynamics in human cortex during visual object recognition , 2015 .

[49]  Timothy J. Andrews,et al.  Internal and External Features of the Face Are Represented Holistically in Face-Selective Regions of Visual Cortex , 2010, The Journal of Neuroscience.

[50]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[51]  C. Granger Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[52]  Ashesh D. Mehta,et al.  Tuning face perception with electrical stimulation of the fusiform gyrus , 2017, Human brain mapping.

[53]  Bruno Rossion,et al.  A steady-state visual evoked potential approach to individual face perception: Effect of inversion, contrast-reversal and temporal dynamics , 2012, NeuroImage.

[54]  Daniel D. Dilks,et al.  Differential selectivity for dynamic versus static information in face-selective cortical regions , 2011, NeuroImage.

[55]  Wendy L. Braje,et al.  Illumination effects in face recognition , 1998, Psychobiology.

[56]  Context may reveal how you feel , 2019, Proceedings of the National Academy of Sciences.

[57]  M. Meng,et al.  Categorical learning revealed in activity pattern of left fusiform cortex , 2017, Human brain mapping.

[58]  Margot J. Taylor,et al.  Inversion and contrast-reversal effects on face processing assessed by MEG , 2006, Brain Research.

[59]  Gaurav Singal,et al.  Lateralization of face processing in the human brain , 2012, Proceedings of the Royal Society B: Biological Sciences.

[60]  Nancy Kanwisher,et al.  Two Critical and Functionally Distinct Stages of Face and Body Perception , 2012, The Journal of Neuroscience.

[61]  Aleix M. Martinez Visual perception of facial expressions of emotion. , 2017, Current Opinion in Psychology.

[62]  Justin P. Haldar,et al.  Canonical granger causality between regions of interest , 2013, NeuroImage.

[63]  Joseph S. Gati,et al.  Differences in perceived shape from shading correlate with activity in early visual areas , 1997, Current Biology.

[64]  D. Pitcher,et al.  The role of the occipital face area in the cortical face perception network , 2011, Experimental Brain Research.

[65]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[66]  F. Tong,et al.  The timing of perceptual decisions for ambiguous face stimuli in the human ventral visual cortex. , 2006, Cerebral cortex.

[67]  W. Drongelen,et al.  Localization of brain electrical activity via linearly constrained minimum variance spatial filtering , 1997, IEEE Transactions on Biomedical Engineering.

[68]  Taiyong Bi,et al.  Function and Structure of Human Left Fusiform Cortex Are Closely Associated with Perceptual Learning of Faces , 2014, Current Biology.

[69]  Scott D. Slotnick,et al.  The Fusiform Face Area , 2013 .

[70]  Gavin Perry,et al.  Localizing evoked and induced responses to faces using magnetoencephalography , 2014, The European journal of neuroscience.

[71]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[72]  Pascal Mamassian,et al.  Prior knowledge of illumination for 3D perception in the human brain , 2010, Proceedings of the National Academy of Sciences.

[73]  N. Kanwisher,et al.  How face perception unfolds over time , 2018, Nature Communications.

[74]  Jia Liu,et al.  Perception of Face Parts and Face Configurations: An fMRI Study , 2010, Journal of Cognitive Neuroscience.

[75]  Moritz Grosse-Wentrup,et al.  Multisubject Learning for Common Spatial Patterns in Motor-Imagery BCI , 2011, Comput. Intell. Neurosci..

[76]  Guy Wallis,et al.  The face-in-the-crowd effect: when angry faces are just cross(es). , 2011, Journal of vision.

[77]  M. Seghier,et al.  A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. , 2003, Brain : a journal of neurology.

[78]  Martin Luessi,et al.  MNE software for processing MEG and EEG data , 2014, NeuroImage.

[79]  J. Haxby,et al.  Neural systems for recognition of familiar faces , 2007, Neuropsychologia.

[80]  Bruno Rossion,et al.  Being BOLD: The neural dynamics of face perception , 2017, Human brain mapping.

[81]  Tessa R. Flack,et al.  Patterns of neural response in face regions are predicted by low-level image properties , 2018, Cortex.

[82]  Anil K. Seth,et al.  The MVGC multivariate Granger causality toolbox: A new approach to Granger-causal inference , 2014, Journal of Neuroscience Methods.

[83]  R. Goebel,et al.  Holistic perception of individual faces in the right middle fusiform gyrus as evidenced by the composite face illusion. , 2010, Journal of vision.

[84]  I. Biederman,et al.  The utility of surface reflectance for the recognition of upright and inverted faces , 2007, Vision Research.

[85]  O. Lipp,et al.  The effect of face inversion on the detection of emotional faces in visual search , 2015, Cognition & emotion.

[86]  A. Treves,et al.  Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain , 2005, Nature Neuroscience.

[87]  Yoko Mano,et al.  Beyond the Memory Mechanism: Person-selective and Nonselective Processes in Recognition of Personally Familiar Faces , 2011, Journal of Cognitive Neuroscience.

[88]  R. Goebel,et al.  Holistic Face Categorization in Higher Order Visual Areas of the Normal and Prosopagnosic Brain: Toward a Non-Hierarchical View of Face Perception , 2011, Front. Hum. Neurosci..

[89]  Patrick Cavanagh,et al.  Recovery of 3D volume from 2-tone images of novel objects , 1998, Cognition.

[90]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[91]  Ethan M. Meyers,et al.  Contextually Evoked Object-Specific Responses in Human Visual Cortex , 2004, Science.

[92]  J. Haxby,et al.  Distinct representations of eye gaze and identity in the distributed human neural system for face perception , 2000, Nature Neuroscience.

[93]  Doris Y. Tsao,et al.  Comparing face patch systems in macaques and humans , 2008, Proceedings of the National Academy of Sciences.

[94]  Vincent Walsh,et al.  Combined TMS and fMRI Reveal Dissociable Cortical Pathways for Dynamic and Static Face Perception , 2014, Current Biology.

[95]  Kenneth F. Valyear,et al.  The fusiform face area is not sufficient for face recognition: Evidence from a patient with dense prosopagnosia and no occipital face area , 2006, Neuropsychologia.

[96]  S. Bentin,et al.  A magnetoencephalographic study of face processing: M170, gamma‐band oscillations and source localization , 2013, Human brain mapping.

[97]  G. Yovel,et al.  Event‐related potential and functional MRI measures of face‐selectivity are highly correlated: A simultaneous ERP‐fMRI investigation , 2010, Human brain mapping.

[98]  Dimitrios Pantazis,et al.  Similarity-Based Fusion of MEG and fMRI Reveals Spatio-Temporal Dynamics in Human Cortex During Visual Object Recognition , 2015, bioRxiv.

[99]  W. Singer,et al.  Neuroelectromagnetic Correlates of Perceptual Closure Processes , 2010, The Journal of Neuroscience.

[100]  Elias B. Issa,et al.  Neural dynamics at successive stages of the ventral visual stream are consistent with hierarchical error signals , 2018, eLife.

[101]  Winrich A Freiwald,et al.  Two areas for familiar face recognition in the primate brain , 2017, Science.

[102]  D. Mumford On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[103]  Nitin Tandon,et al.  Network dynamics of human face perception , 2017, PloS one.

[104]  T. Allison,et al.  Electrophysiological Studies of Face Perception in Humans , 1996, Journal of Cognitive Neuroscience.

[105]  P. Sinha,et al.  Contribution of Color to Face Recognition , 2002, Perception.

[106]  Radoslaw Martin Cichy,et al.  Resolving human object recognition in space and time , 2014, Nature Neuroscience.

[107]  J. Haxby,et al.  Neural response to the visual familiarity of faces , 2006, Brain Research Bulletin.

[108]  Greg O. Horne,et al.  Controlling low-level image properties: The SHINE toolbox , 2010, Behavior research methods.

[109]  Galit Yovel,et al.  Two neural pathways of face processing: A critical evaluation of current models , 2015, Neuroscience & Biobehavioral Reviews.

[110]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[111]  Elias B. Issa,et al.  Precedence of the Eye Region in Neural Processing of Faces , 2012, The Journal of Neuroscience.

[112]  D. Whitney,et al.  Tracking the affective state of unseen persons , 2019, Proceedings of the National Academy of Sciences.

[113]  Xue Wang,et al.  Estimating Granger causality after stimulus onset: A cautionary note , 2008, NeuroImage.

[114]  Pim Mostert,et al.  Differential temporal dynamics during visual imagery and perception , 2018, eLife.