Revealing the Semantic Association between Perception of Scenes and Significant Objects by Representational Similarity Analysis

'Significant' objects contribute greatly to scene recognition. The lateral occipital complex (LOC), parahippocampal place area (PPA), and retrosplenial cortex (RSC) play a crucial role in the cognitive processing of objects and scenes. However, the associated mechanism between objects and scenes remains unclear. In this study, four categories of scene images and four types of significant objects were designed as stimuli. Representational similarity analysis (RSA) of functional magnetic resonance imaging (fMRI) data showed that correlation coefficients of the activity patterns for objects and scenes were significantly positive in the LOC and PPA. Compared to the out-of-scene objects, the correlation strengths for within-scene objects were significantly greater in the PPA and two subregions of the LOC: the lateral occipital area (LO), and posterior fusiform area (PF). Further correlation analyses showed that the scene-object correlations were different for indoor and outdoor scenes in the LO, pF and PPA. Semantic associations were represented in the LO and pF, while the PPA was involved in semantic correlations and spatial characteristics, which were sensitive to the openness of scenes. However, these trends were not observed in the RSC, suggesting that it is not recruited to process semantic associations between scenes and objects. Our findings provide an understanding of the neural mechanism of scene recognition.

[1]  Russell A. Epstein.  Parahippocampal and retrosplenial contributions to human spatial navigation , 2008, Trends in Cognitive Sciences.

[2]  J. S. Guntupalli,et al.  The Representation of Biological Classes in the Human Brain , 2012, The Journal of Neuroscience.

[3]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[4]  Paul T. Sowden,et al.  The use of visual information in natural scenes , 2005 .

[5]  Jonathan S. Cant,et al.  Object Ensemble Processing in Human Anterior-Medial Ventral Visual Cortex , 2012, The Journal of Neuroscience.

[6]  Michelle R. Greene,et al.  Recognition of natural scenes from global properties: Seeing the forest without representing the trees , 2009, Cognitive Psychology.

[7]  Richard N. Aslin,et al.  Decoding the infant mind: Multivariate pattern analysis (MVPA) using fNIRS , 2017, PloS one.

[8]  Russell A. Epstein,et al.  Constructing scenes from objects in human occipitotemporal cortex , 2011, Nature Neuroscience.

[9]  M. D’Esposito,et al.  An Area within Human Ventral Cortex Sensitive to “Building” Stimuli Evidence and Implications , 1998, Neuron.

[10]  Guillaume A. Rousselet,et al.  Processing scene context: Fast categorization and object interference , 2007, Vision Research.

[11]  Jonathan S. Cant,et al.  Cerebral Cortex Advance Access published April 28, 2006 Attention to Form or Surface Properties Modulates Different Regions of Human , 2022 .

[12]  A. Oliva,et al.  Diagnostic Colors Mediate Scene Recognition , 2000, Cognitive Psychology.

[13]  Jodi L. Davenport,et al.  Scene Consistency in Object and Background Perception , 2004, Psychological science.

[14]  M. Fabre-Thorpe,et al.  Incongruent object/context relationships in visual scenes: Where are they processed in the brain? , 2014, Brain and Cognition.

[15]  Gaoyan Zhang,et al.  Decoding facial expressions based on face‐selective and motion‐sensitive areas , 2017, Human brain mapping.

[16]  Heinrich H. Bülthoff,et al.  Categorization of natural scenes: Local versus global information and the role of color , 2007, TAP.

[17]  N. Kanwisher,et al.  Cortical Regions Involved in Perceiving Object Shape , 2000, The Journal of Neuroscience.

[18]  E. Maguire,et al.  Exploring the parahippocampal cortex response to high and low spatial frequency spaces , 2012, Neuroreport.

[19]  A. Norcia,et al.  A Representational Similarity Analysis of the Dynamics of Object Processing Using Single-Trial EEG Classification , 2015, PloS one.

[20]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[21]  Dwight J. Kravitz,et al.  Real-World Scene Representations in High-Level Visual Cortex: It's the Spaces More Than the Places , 2011, The Journal of Neuroscience.

[22]  Soojin Park,et al.  Disentangling Scene Content from Spatial Boundary: Complementary Roles for the Parahippocampal Place Area and Lateral Occipital Complex in Representing Real-World Scenes , 2011, The Journal of Neuroscience.

[23]  Dimitrios Pantazis,et al.  Multivariate pattern analysis of MEG and EEG: A comparison of representational structure in time and space , 2016, NeuroImage.

[24]  Geoffrey Karl Aguirre,et al.  Continuous carry-over designs for fMRI , 2007, NeuroImage.

[25]  J. Devlin,et al.  Triple Dissociation of Faces, Bodies, and Objects in Extrastriate Cortex , 2009, Current Biology.

[26]  Frederick A A Kingdom,et al.  Comparison of sensitivity to color changes in natural and phase-scrambled scenes. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[27]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[28]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[29]  James V. Haxby,et al.  Multivariate pattern analysis of fMRI: The early beginnings , 2012, NeuroImage.

[30]  Natalia Y. Bilenko,et al.  The “Parahippocampal Place Area” Responds Preferentially to High Spatial Frequencies in Humans and Monkeys , 2011, PLoS biology.

[31]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[32]  T. Carlson,et al.  Patterns of Activity in the Categorical Representations of Objects , 2003 .

[33]  P. Perona,et al.  What do we perceive in a glance of a real-world scene? , 2007, Journal of vision.

[34]  T. Fukami,et al.  Brain Activation in Response to Disgustful Face Images with Different Backgrounds , 2009 .

[35]  Russell A. Epstein,et al.  Decoding the Representation of Multiple Simultaneous Objects in Human Occipitotemporal Cortex , 2009, Current Biology.

[36]  Dwight J. Kravitz,et al.  Deconstructing visual scenes in cortex: gradients of object and spatial layout information. , 2013, Cerebral cortex.

[37]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[38]  Drew Linsley,et al.  Evidence for participation by object-selective visual cortex in scene category judgments. , 2014, Journal of vision.

[39]  Jiye G. Kim,et al.  Where do objects become scenes? , 2011, Cerebral cortex.

[40]  M. Goodale,et al.  Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. , 2003, Brain : a journal of neurology.

[41]  S. Edelman,et al.  Cue-Invariant Activation in Object-Related Areas of the Human Occipital Lobe , 1998, Neuron.