Deconstructing visual scenes in cortex: gradients of object and spatial layout information.

Real-world visual scenes are complex cluttered, and heterogeneous stimuli engaging scene- and object-selective cortical regions including parahippocampal place area (PPA), retrosplenial complex (RSC), and lateral occipital complex (LOC). To understand the unique contribution of each region to distributed scene representations, we generated predictions based on a neuroanatomical framework adapted from monkey and tested them using minimal scenes in which we independently manipulated both spatial layout (open, closed, and gradient) and object content (furniture, e.g., bed, dresser). Commensurate with its strong connectivity with posterior parietal cortex, RSC evidenced strong spatial layout information but no object information, and its response was not even modulated by object presence. In contrast, LOC, which lies within the ventral visual pathway, contained strong object information but no background information. Finally, PPA, which is connected with both the dorsal and the ventral visual pathway, showed information about both objects and spatial backgrounds and was sensitive to the presence or absence of either. These results suggest that 1) LOC, PPA, and RSC have distinct representations, emphasizing different aspects of scenes, 2) the specific representations in each region are predictable from their patterns of connectivity, and 3) PPA combines both spatial layout and object information as predicted by connectivity.

[1]  Dwight J. Kravitz,et al.  Real-World Scene Representations in High-Level Visual Cortex: It's the Spaces More Than the Places , 2011, The Journal of Neuroscience.

[2]  Leslie G. Ungerleider,et al.  Object representations in the temporal cortex of monkeys and humans as revealed by functional magnetic resonance imaging. , 2009, Journal of neurophysiology.

[3]  Magdalena G. Wutte,et al.  Modality-Independent Coding of Spatial Layout in the Human Brain , 2011, Current Biology.

[4]  Soojin Park,et al.  Disentangling Scene Content from Spatial Boundary: Complementary Roles for the Parahippocampal Place Area and Lateral Occipital Complex in Representing Real-World Scenes , 2011, The Journal of Neuroscience.

[5]  Russell A. Epstein,et al.  Where Am I Now? Distinct Roles for Parahippocampal and Retrosplenial Cortices in Place Recognition , 2007, The Journal of Neuroscience.

[6]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[7]  Robert W. Kentridge,et al.  Separate channels for processing form, texture, and color: evidence from FMRI adaptation and visual object agnosia. , 2010, Cerebral cortex.

[8]  Russell A. Epstein,et al.  Decoding the Representation of Multiple Simultaneous Objects in Human Occipitotemporal Cortex , 2009, Current Biology.

[9]  D. Amaral,et al.  Macaque monkey retrosplenial cortex: II. Cortical afferents , 2003, The Journal of comparative neurology.

[10]  M. Potter,et al.  Recognition memory for a rapid sequence of pictures. , 1969, Journal of experimental psychology.

[11]  Gabriele Janzen,et al.  Selective neural representation of objects relevant for navigation , 2004, Nature Neuroscience.

[12]  Dwight J. Kravitz,et al.  High-level visual object representations are constrained by position. , 2010, Cerebral cortex.

[13]  Robert Desimone,et al.  Cortical connections of area V4 in the macaque. , 2000, Cerebral cortex.

[14]  Dirk B. Walther,et al.  Natural Scene Categories Revealed in Distributed Patterns of Activity in the Human Brain , 2009, The Journal of Neuroscience.

[15]  Guillaume A. Rousselet,et al.  Processing scene context: Fast categorization and object interference , 2007, Vision Research.

[16]  M. Bar,et al.  Cortical Analysis of Visual Context , 2003, Neuron.

[17]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[18]  N. Kanwisher,et al.  How Distributed Is Visual Category Information in Human Occipito-Temporal Cortex? An fMRI Study , 2002, Neuron.

[19]  M. D’Esposito,et al.  Topographical disorientation: a synthesis and taxonomy. , 1999, Brain : a journal of neurology.

[20]  Russell A. Epstein,et al.  Constructing scenes from objects in human occipitotemporal cortex , 2011, Nature Neuroscience.

[21]  Russell A. Epstein,et al.  Visual scene processing in familiar and unfamiliar environments. , 2007, Journal of neurophysiology.

[22]  Jiye G. Kim,et al.  Where do objects become scenes? , 2011, Cerebral cortex.

[23]  Z Kourtzi,et al.  Representation of Perceived Object Shape by the Human Lateral Occipital Complex , 2001, Science.

[24]  Nancy Kanwisher,et al.  The distribution of category and location information across object-selective regions in human visual cortex , 2008, Proceedings of the National Academy of Sciences.

[25]  Russell A. Epstein Parahippocampal and retrosplenial contributions to human spatial navigation , 2008, Trends in Cognitive Sciences.

[26]  D. Amaral,et al.  Macaque monkey retrosplenial cortex: III. Cortical efferents , 2003, The Journal of comparative neurology.

[27]  E. Maguire The retrosplenial contribution to human navigation: a review of lesion and neuroimaging findings. , 2001, Scandinavian journal of psychology.

[28]  J. Devlin,et al.  Triple Dissociation of Faces, Bodies, and Objects in Extrastriate Cortex , 2009, Current Biology.

[29]  N. Kriegeskorte,et al.  Revealing representational content with pattern-information fMRI--an introductory guide. , 2009, Social cognitive and affective neuroscience.

[30]  Soojin Park,et al.  Different roles of the parahippocampal place area (PPA) and retrosplenial cortex (RSC) in panoramic scene perception , 2009, NeuroImage.

[31]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[32]  Russell A. Epstein,et al.  The Parahippocampal Place Area Recognition, Navigation, or Encoding? , 1999, Neuron.

[33]  Emily J. Ward,et al.  How reliable are visual context effects in the parahippocampal place area? , 2010, Cerebral cortex.

[34]  M. Bar,et al.  The parahippocampal cortex mediates spatial and nonspatial associations. , 2007, Cerebral cortex.

[35]  M. D’Esposito,et al.  The parahippocampus subserves topographical learning in man , 1996, NeuroImage.

[36]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[37]  H. Intraub,et al.  Beyond the Edges of a View: Boundary Extension in Human Scene-Selective Visual Cortex , 2007, Neuron.

[38]  Ravi S. Menon,et al.  Behavioral and Neuroimaging Evidence for a Contribution of Color and Texture Information to Scene Classification in a Patient with Visual Form Agnosia , 2004, Journal of Cognitive Neuroscience.

[39]  S. Becker,et al.  Remembering the past and imagining the future: a neural model of spatial memory and imagery. , 2007, Psychological review.

[40]  D. Hassabis,et al.  Using Imagination to Understand the Neural Basis of Episodic Memory , 2007, The Journal of Neuroscience.

[41]  E. Maguire,et al.  What does the retrosplenial cortex do? , 2009, Nature Reviews Neuroscience.

[42]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[43]  Daniel D. Dilks,et al.  Mirror-Image Sensitivity and Invariance in Object and Scene Processing Pathways , 2011, The Journal of Neuroscience.

[44]  M. Bar,et al.  Scenes Unseen: The Parahippocampal Cortex Intrinsically Subserves Contextual Associations, Not Scenes or Places Per Se , 2008, The Journal of Neuroscience.

[45]  K. Saleem,et al.  Differential connections of the perirhinal and parahippocampal cortex with the orbital and medial prefrontal networks in macaque monkeys , 2005, The Journal of comparative neurology.

[46]  Dwight J. Kravitz,et al.  A new neural framework for visuospatial processing , 2011, Nature Reviews Neuroscience.

[47]  Russell A. Epstein,et al.  Differential parahippocampal and retrosplenial involvement in three types of visual scene recognition. , 2006, Cerebral cortex.

[48]  D. Pandya,et al.  Cingulate cortex of the rhesus monkey: II. Cortical afferents , 1987, The Journal of comparative neurology.

[49]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[50]  T. Kimura,et al.  Mental navigation in humans is processed in the anterior bank of the parieto-occipital sulcus , 2002, Neuroscience Letters.

[51]  Jonathan S. Cant,et al.  Scratching Beneath the Surface: New Insights into the Functional Properties of the Lateral Occipital Area and Parahippocampal Place Area , 2011, The Journal of Neuroscience.

[52]  Dwight J. Kravitz,et al.  Cortical representations of bodies and faces are strongest in their commonly experienced configurations , 2010, Nature Neuroscience.

[53]  M. Bar,et al.  Early onset of neural synchronization in the contextual associations network , 2011, Proceedings of the National Academy of Sciences.

[54]  M. Goodale,et al.  Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. , 2003, Brain : a journal of neurology.

[55]  Yasufumi Tanaka,et al.  Heading Disorientation: A New Test and a Possible Underlying Mechanism , 2010, European Neurology.

[56]  Leslie G. Ungerleider,et al.  Scene-Selective Cortical Regions in Human and Nonhuman Primates , 2011, The Journal of Neuroscience.

[57]  Eleanor A Maguire,et al.  A New Role for the Parahippocampal Cortex in Representing Space , 2011, The Journal of Neuroscience.

[58]  Li Fei-Fei,et al.  Simple line drawings suffice for functional MRI decoding of natural scene categories , 2011, Proceedings of the National Academy of Sciences.

[59]  N. Kanwisher,et al.  The lateral occipital complex and its role in object recognition , 2001, Vision Research.

[60]  M. Potter Short-term conceptual memory for pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[61]  P. C. Murphy,et al.  Cerebral Cortex , 2017, Cerebral Cortex.