Prevalence of Selectivity for Mirror-Symmetric Views of Faces in the Ventral and Dorsal Visual Pathways

Although the ability to recognize faces and objects from a variety of viewpoints is crucial to our everyday behavior, the underlying cortical mechanisms are not well understood. Recently, neurons in a face-selective region of the monkey temporal cortex were reported to be selective for mirror-symmetric viewing angles of faces as they were rotated in depth (Freiwald and Tsao, 2010). This property has been suggested to constitute a key computational step in achieving full view-invariance. Here, we measured functional magnetic resonance imaging activity in nine observers as they viewed upright or inverted faces presented at five different angles (−60, −30, 0, 30, and 60°). Using multivariate pattern analysis, we show that sensitivity to viewpoint mirror symmetry is widespread in the human visual system. The effect was observed in a large band of higher order visual areas, including the occipital face area, fusiform face area, lateral occipital cortex, mid fusiform, parahippocampal place area, and extending superiorly to encompass dorsal regions V3A/B and the posterior intraparietal sulcus. In contrast, early retinotopic regions V1–hV4 failed to exhibit sensitivity to viewpoint symmetry, as their responses could be largely explained by a computational model of low-level visual similarity. Our findings suggest that selectivity for mirror-symmetric viewing angles may constitute an intermediate-level processing step shared across multiple higher order areas of the ventral and dorsal streams, setting the stage for complete viewpoint-invariant representations at subsequent levels of visual processing.

[1]  K. Grill-Spector The neural basis of object perception , 2003, Current Opinion in Neurobiology.

[2]  M. D’Esposito,et al.  An Area within Human Ventral Cortex Sensitive to “Building” Stimuli Evidence and Implications , 1998, Neuron.

[3]  A. Dale,et al.  High‐resolution intersubject averaging and a coordinate system for the cortical surface , 1999, Human brain mapping.

[4]  T. Valentine Upside-down faces: a review of the effect of inversion upon face recognition. , 1988, British journal of psychology.

[5]  Leslie G. Ungerleider,et al.  The Effect of Face Inversion on Activity in Human Neural Systems for Face and Object Perception , 1999, Neuron.

[6]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[7]  Doris Y. Tsao,et al.  Functional Compartmentalization and Viewpoint Generalization Within the Macaque Face-Processing System , 2010, Science.

[8]  D. Pitcher,et al.  The role of the occipital face area in the cortical face perception network , 2011, Experimental Brain Research.

[9]  S. Ullman Three-dimensional object recognition based on the combination of views , 1998, Cognition.

[10]  K. Nakayama,et al.  The effect of face inversion on the human fusiform face area , 1998, Cognition.

[11]  J. Culham,et al.  The human dorsal stream adapts to real actions and 3D shape processing: a functional magnetic resonance imaging study. , 2008, Journal of neurophysiology.

[12]  M. Harries,et al.  Viewer-centred and object-centred coding of heads in the macaque temporal cortex , 2004, Experimental Brain Research.

[13]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[14]  G. Glover,et al.  Retinotopic organization in human visual cortex and the spatial precision of functional MRI. , 1997, Cerebral cortex.

[15]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[16]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[17]  S. Kastner,et al.  Two hierarchically organized neural systems for object information in human visual cortex , 2008, Nature Neuroscience.

[18]  R. Goebel,et al.  Individual faces elicit distinct response patterns in human anterior temporal cortex , 2007, Proceedings of the National Academy of Sciences.

[19]  Leslie G. Ungerleider,et al.  Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys , 1982, Behavioural Brain Research.

[20]  T. Allison,et al.  Differential Sensitivity of Human Visual Cortex to Faces, Letterstrings, and Textures: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.

[21]  T. Poggio,et al.  A network that learns to recognize three-dimensional objects , 1990, Nature.

[22]  R. Yin Looking at Upside-down Faces , 1969 .

[23]  Michael S. Pratte,et al.  Decoding patterns of human brain activity. , 2012, Annual review of psychology.

[24]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[25]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[26]  M. D’Esposito,et al.  Stimulus inversion and the responses of face and object-sensitive cortical areas. , 1999, Neuroreport.

[27]  Keiji Tanaka,et al.  Inferotemporal cortex and object vision. , 1996, Annual review of neuroscience.

[28]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[29]  C. Gross,et al.  Neural representations of faces and body parts in macaque and human cortex: a comparative FMRI study. , 2009, Journal of neurophysiology.

[30]  J. Mazziotta,et al.  A Locus in Human Extrastriate Cortex for Visual Shape Analysis , 1997, Journal of Cognitive Neuroscience.

[31]  Stanislas Dehaene,et al.  Why do children make mirror errors in reading? Neural correlates of mirror invariance in the visual word form area , 2010, NeuroImage.

[32]  Leslie G. Ungerleider,et al.  Distributed representation of objects in the human ventral visual pathway. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[34]  J W Belliveau,et al.  Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. , 1995, Science.

[35]  Kalanit Grill-Spector,et al.  The representation of object viewpoint in human visual cortex , 2009, NeuroImage.

[36]  D. Perrett,et al.  Evidence accumulation in cell populations responsive to faces: an account of generalisation of recognition without mental transformations , 1998, Cognition.

[37]  Lawrence L. Wald,et al.  Accurate prediction of V1 location from cortical folds in a surface coordinate system , 2008, NeuroImage.

[38]  T. Poggio,et al.  The importance of symmetry and virtual views in three-dimensional object recognition , 1994, Current Biology.

[39]  J. Haynes Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .

[40]  Alexander Borst,et al.  How does Nature Program Neuron Types? , 2008, Front. Neurosci..

[41]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[42]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[43]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[44]  Brian A. Wandell,et al.  Predominantly extra-retinotopic cortical response to pattern symmetry , 2005, NeuroImage.

[45]  C R Olson,et al.  Mirror-image confusion in single neurons of the macaque inferotemporal cortex. , 2000, Science.

[46]  Thomas Serre,et al.  Realistic Modeling of Simple and Complex Cell Tuning in the HMAX Model, and Implications for Invariant Object Recognition in Cortex , 2004 .

[47]  Russell A. Epstein,et al.  Cortical correlates of face and scene inversion: A comparison , 2006, Neuropsychologia.

[48]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[49]  Wim Vanduffel,et al.  Symmetry activates extrastriate visual cortex in human and nonhuman primates. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[50]  Martin A. Riedmiller,et al.  Computational object recognition: a biologically motivated approach , 2009, Biological Cybernetics.

[51]  Marlene Behrmann,et al.  Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis , 2011, Proceedings of the National Academy of Sciences.

[52]  S. Edelman,et al.  Differential Processing of Objects under Various Viewing Conditions in the Human Lateral Occipital Complex , 1999, Neuron.

[53]  Gilles Pourtois,et al.  Portraits or People? Distinct Representations of Face Identity in the Human Visual Cortex , 2005, Journal of Cognitive Neuroscience.

[54]  M. Tarr,et al.  The Fusiform Face Area is Part of a Network that Processes Faces at the Individual Level , 2000, Journal of Cognitive Neuroscience.

[55]  John H. R. Maunsell,et al.  Shape selectivity in primate lateral intraparietal cortex , 1998, Nature.

[56]  G. Yovel,et al.  Hierarchical Processing of Face Viewpoint in Human Visual Cortex , 2012, The Journal of Neuroscience.

[57]  Doris Y. Tsao,et al.  Comparing face patch systems in macaques and humans , 2008, Proceedings of the National Academy of Sciences.

[58]  H H Bülthoff,et al.  Psychophysical support for a two-dimensional view interpolation theory of object recognition. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[59]  Isabel Gauthier,et al.  Three-dimensional object recognition is viewpoint dependent , 1998, Nature Neuroscience.

[60]  R. Henson,et al.  BOLD repetition decreases in object-responsive ventral visual areas depend on spatial attention. , 2004, Journal of neurophysiology.

[61]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[62]  Alice J. O'Toole,et al.  Dissociable Neural Patterns of Facial Identity across Changes in Viewpoint , 2010, Journal of Cognitive Neuroscience.

[63]  Daniel D. Dilks,et al.  Mirror-Image Sensitivity and Invariance in Object and Scene Processing Pathways , 2011, The Journal of Neuroscience.

[64]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.