Evidence for participation by object-selective visual cortex in scene category judgments.

Scene recognition is a core function of the visual system, drawing both on scenes' intrinsic global features, prominently their spatial properties, and on the identities of the objects scenes contain. Neuroimaging and neuropsychological studies have associated spatial property-based scene categorization with parahippocampal cortex, while processing of scene-relevant object information is associated with the lateral occipital complex (LOC), wherein activity patterns distinguish between categories of standalone objects and those embedded in scenes. However, despite the importance of objects to scene categorization and the role of LOC in processing them, damage or disruption to LOC that hampers object recognition has been shown to improve scene categorization. To address this paradox, we used functional magnetic resonance imaging (fMRI) to directly assess the contributions of LOC and the parahippocampal place area (PPA) to category judgments of indoor scenes that were devoid of objective identity signals. Observers were alternately cued to base judgments on scenes' objects or spatial properties. In both LOC and PPA, multivoxel activity patterns better decoded judgments based on their typically associated features: LOC more accurately decoded object-based judgments, while PPA more accurately decoded spatial property-based judgments. The cue contingency of LOC decoding accuracy indicates that it was not an outcome of feedback from judgments and is instead consistent with dependency of judgments on the output of object processing pathways in which LOC participates.

[1]  Russell A. Epstein Parahippocampal and retrosplenial contributions to human spatial navigation , 2008, Trends in Cognitive Sciences.

[2]  Russell A. Epstein,et al.  Neural responses to visual scenes reveals inconsistencies between fMRI adaptation and multivoxel pattern analysis , 2012, Neuropsychologia.

[3]  Paul T. Sowden,et al.  The use of visual information in natural scenes , 2005 .

[4]  M. Potter,et al.  Recognition memory for a rapid sequence of pictures. , 1969, Journal of experimental psychology.

[5]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[6]  Drew Linsley,et al.  Encoding-Stage Crosstalk Between Object- and Spatial Property-Based Scene Processing Pathways. , 2015, Cerebral cortex.

[7]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[8]  M. Potter Short-term conceptual memory for pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[9]  Russell A. Epstein,et al.  Neuropsychological evidence for a topographical learning mechanism in parahippocampal cortex , 2001, Cognitive neuropsychology.

[10]  Heinrich H. Bülthoff,et al.  Categorization of natural scenes: Local versus global information and the role of color , 2007, TAP.

[11]  Marcelo Gomes Mattar,et al.  de Bruijn cycles for neural decoding , 2011, NeuroImage.

[12]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[13]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[14]  Russell A. Epstein,et al.  The Parahippocampal Place Area Recognition, Navigation, or Encoding? , 1999, Neuron.

[15]  Jitendra Malik,et al.  When is scene identification just texture recognition? , 2004, Vision Research.

[16]  N. Kanwisher,et al.  Only some spatial patterns of fMRI response are read out in task performance , 2007, Nature Neuroscience.

[17]  Li Fei-Fei,et al.  Neural mechanisms of rapid natural scene categorization in human visual cortex , 2009, Nature.

[18]  Nancy Kanwisher,et al.  An algorithmic method for functionally defining regions of interest in the ventral visual pathway , 2012, NeuroImage.

[19]  M. Bar Visual objects in context , 2004, Nature Reviews Neuroscience.

[20]  P. Perona,et al.  What do we perceive in a glance of a real-world scene? , 2007, Journal of vision.

[21]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[22]  R. VanRullen,et al.  Faces in the cloud: Fourier power spectrum biases ultrarapid face detection. , 2008, Journal of vision.

[23]  M. Cherrier,et al.  Agnosia for scenes in topographagnosia , 2003, Neuropsychologia.

[24]  M. D’Esposito,et al.  Topographical disorientation: a synthesis and taxonomy. , 1999, Brain : a journal of neurology.

[25]  M. Goodale,et al.  Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. , 2003, Brain : a journal of neurology.

[26]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[27]  M. Potter Meaning in visual search. , 1975, Science.

[28]  I. Biederman Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[29]  I. Biederman,et al.  Scene perception: Detecting and judging objects undergoing relational violations , 1982, Cognitive Psychology.

[30]  Rachel A Diana,et al.  High‐resolution multi‐voxel pattern analysis of category selectivity in the medial temporal lobes , 2008, Hippocampus.

[31]  Irving Biederman,et al.  Object search in nonscene displays. , 1988 .

[32]  Russell A Poldrack,et al.  Hemispheric asymmetries and individual differences in visual concept learning as measured by functional MRI , 2000, Neuropsychologia.

[33]  Ravi S. Menon,et al.  Behavioral and Neuroimaging Evidence for a Contribution of Color and Texture Information to Scene Classification in a Patient with Visual Form Agnosia , 2004, Journal of Cognitive Neuroscience.

[34]  M. Bar,et al.  The parahippocampal cortex mediates spatial and nonspatial associations. , 2007, Cerebral cortex.

[35]  Russell A. Epstein,et al.  Constructing scenes from objects in human occipitotemporal cortex , 2011, Nature Neuroscience.

[36]  S. Edelman,et al.  Cue-Invariant Activation in Object-Related Areas of the Human Occipital Lobe , 1998, Neuron.

[37]  A. Sirigu,et al.  Pure Topographical Disorientation: A Definition and Anatomical Basis , 1987, Cortex.

[38]  M. Giese,et al.  Flexible Coding for Categorical Decisions in the Human Brain , 2007, The Journal of Neuroscience.

[39]  F. Gosselin,et al.  Why do we SLIP to the basic level? Computational constraints and their implementation. , 2001, Psychological review.

[40]  M. D’Esposito,et al.  An Area within Human Ventral Cortex Sensitive to “Building” Stimuli Evidence and Implications , 1998, Neuron.

[41]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[42]  Jodi L. Davenport,et al.  Scene Consistency in Object and Background Perception , 2004, Psychological science.

[43]  Murray Grossman,et al.  The Neural Basis for Category-Specific Knowledge: An fMRI Study , 2002, NeuroImage.

[44]  Soojin Park,et al.  Different roles of the parahippocampal place area (PPA) and retrosplenial cortex (RSC) in panoramic scene perception , 2009, NeuroImage.

[45]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[46]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[47]  Stephen M. Kosslyn,et al.  Do separate processes identify objects as exemplars versus members of basic-level categories? Evidence from hemispheric specialization , 2003, Brain and Cognition.

[48]  P. Rondot,et al.  Loss of Topographic Memory with Learning Deficits , 1980, Cortex.

[49]  Jennifer K. E. Steeves,et al.  TMS to the Lateral Occipital Cortex Disrupts Object Processing but Facilitates Scene Processing , 2011, Journal of Cognitive Neuroscience.

[50]  A. Oliva,et al.  Diagnostic Colors Mediate Scene Recognition , 2000, Cognitive Psychology.

[51]  Sean P. MacEvoy,et al.  Joint neuronal tuning for object form and position in the human lateral occipital complex , 2012, NeuroImage.

[52]  L. Deouell,et al.  ERP evidence for context congruity effects during simultaneous object–scene processing , 2010, Neuropsychologia.

[53]  Dwight J. Kravitz,et al.  Deconstructing visual scenes in cortex: gradients of object and spatial layout information. , 2013, Cerebral cortex.

[54]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[55]  Eleanor A Maguire,et al.  A New Role for the Parahippocampal Cortex in Representing Space , 2011, The Journal of Neuroscience.

[56]  Li Fei-Fei,et al.  Simple line drawings suffice for functional MRI decoding of natural scene categories , 2011, Proceedings of the National Academy of Sciences.

[57]  A. Friedman Framing pictures: the role of knowledge in automatized encoding and memory for gist. , 1979, Journal of experimental psychology. General.

[58]  K. Grill-Spector The neural basis of object perception , 2003, Current Opinion in Neurobiology.

[59]  J. Henderson,et al.  High-level scene perception. , 1999, Annual review of psychology.

[60]  T. Carlson,et al.  Patterns of Activity in the Categorical Representations of Objects , 2003, Journal of Cognitive Neuroscience.

[61]  I. Biederman Perceiving Real-World Scenes , 1972, Science.

[62]  B. Tversky,et al.  Categories of environmental scenes , 1983, Cognitive Psychology.

[63]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[64]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[65]  Michelle R. Greene,et al.  PSYCHOLOGICAL SCIENCE Research Article The Briefest of Glances The Time Course of Natural Scene Understanding , 2022 .

[66]  J. Devlin,et al.  Triple Dissociation of Faces, Bodies, and Objects in Extrastriate Cortex , 2009, Current Biology.

[67]  Geoffrey M Boynton,et al.  The Representation of Behavioral Choice for Motion in Human Visual Cortex , 2007, The Journal of Neuroscience.

[68]  Jodi L. Davenport Consistency effects between objects in scenes , 2007, Memory & cognition.

[69]  K. Grill-Spector,et al.  Relating retinotopic and object-selective responses in human lateral occipital cortex. , 2008, Journal of neurophysiology.

[70]  P. de Graef,et al.  Perceptual effects of scene context on object identification , 1990, Psychological research.

[71]  J. Duncan,et al.  Top-Down Activation of Shape-Specific Population Codes in Visual Cortex during Mental Imagery , 2009, The Journal of Neuroscience.

[72]  Russell A. Epstein,et al.  Differential parahippocampal and retrosplenial involvement in three types of visual scene recognition. , 2006, Cerebral cortex.

[73]  Radoslaw Martin Cichy,et al.  Imagery and perception share cortical representations of content and location. , 2012, Cerebral Cortex.

[74]  Russell A. Epstein,et al.  Decoding the Representation of Multiple Simultaneous Objects in Human Occipitotemporal Cortex , 2009, Current Biology.

[75]  Michelle R. Greene,et al.  Recognition of natural scenes from global properties: Seeing the forest without representing the trees , 2009, Cognitive Psychology.

[76]  Dirk B. Walther,et al.  Natural Scene Categories Revealed in Distributed Patterns of Activity in the Human Brain , 2009, The Journal of Neuroscience.

[77]  Guillaume A. Rousselet,et al.  Processing scene context: Fast categorization and object interference , 2007, Vision Research.

[78]  M. Bar,et al.  The role of the parahippocampal cortex in cognition , 2013, Trends in Cognitive Sciences.

[79]  Leslie G. Ungerleider,et al.  Distributed representation of objects in the human ventral visual pathway. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[80]  N. Kanwisher,et al.  The lateral occipital complex and its role in object recognition , 2001, Vision Research.

[81]  N. Kanwisher,et al.  Cortical Regions Involved in Perceiving Object Shape , 2000, The Journal of Neuroscience.

[82]  Dwight J. Kravitz,et al.  Real-World Scene Representations in High-Level Visual Cortex: It's the Spaces More Than the Places , 2011, The Journal of Neuroscience.

[83]  Soojin Park,et al.  Disentangling Scene Content from Spatial Boundary: Complementary Roles for the Parahippocampal Place Area and Lateral Occipital Complex in Representing Real-World Scenes , 2011, The Journal of Neuroscience.