Investigating the Impact of the Missing Significant Objects in Scene Recognition Using Multivariate Pattern Analysis

Significant objects in a scene can make a great contribution to scene recognition. Besides the three scene-selective regions: parahippocampal place area (PPA), retrosplenial complex (RSC), and occipital place area (OPA), some neuroimaging studies have shown that the lateral occipital complex (LOC) is also engaged in scene recognition processing. In this study, the multivariate pattern analysis was adopted to explore the object-scene association in scene recognition when different amounts of significant objects were masked. The scene classification only succeeded in the intact scene in the ROIs. In addition, the average signal intensity in LOC [including the lateral occipital cortex (LO) and the posterior fusiform area (pF)] decreased when there were masked objects, but such a decrease was not observed in scene-selective regions. These results suggested that LOC was sensitive to the loss of significant objects and mainly involved in scene recognition by the object-scene semantic association. The performance of the scene-selective areas may be mainly due to the fact that they responded to the change of the scene's entire attribute, such as the spatial information, when they were employed in the scene recognition processing. These findings further enrich our knowledge of the significant objects' influence on the activation pattern during the process of scene recognition.

[1]  Baolin Liu,et al.  Investigating the Brain Neural Mechanism when Signature Objects were Masked during a Scene Categorization Task using Functional MRI , 2018, Neuroscience.

[2]  Moshe Bar,et al.  Famous faces activate contextual associations in the parahippocampal cortex. , 2008, Cerebral cortex.

[3]  T. Carlson,et al.  Patterns of Activity in the Categorical Representations of Objects , 2003, Journal of Cognitive Neuroscience.

[4]  Russell A. Poldrack,et al.  What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis , 2014, NeuroImage.

[5]  G. Yovel,et al.  Hierarchical Processing of Face Viewpoint in Human Visual Cortex , 2012, The Journal of Neuroscience.

[6]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[7]  Aude Oliva,et al.  Interaction envelope: Local spatial representations of objects at all scales in scene-selective regions , 2015, NeuroImage.

[8]  Geraint Rees,et al.  Pattern classification using functional magnetic resonance imaging. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[9]  Russell A. Epstein,et al.  Position selectivity in scene- and object-responsive occipitotemporal regions. , 2007, Journal of neurophysiology.

[10]  Lars Muckli,et al.  Decoding Sound and Imagery Content in Early Visual Cortex , 2014, Current Biology.

[11]  Li Fei-Fei,et al.  Neural mechanisms of rapid natural scene categorization in human visual cortex , 2009, Nature.

[12]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[13]  Radoslaw Martin Cichy,et al.  Imagery and perception share cortical representations of content and location. , 2012, Cerebral Cortex.

[14]  Li Fei-Fei,et al.  Simple line drawings suffice for functional MRI decoding of natural scene categories , 2011, Proceedings of the National Academy of Sciences.

[15]  Russell A. Epstein,et al.  Common Neural Representations for Visually Guided Reorientation and Spatial Imagery , 2016, Cerebral cortex.

[16]  M. Bar,et al.  The parahippocampal cortex mediates spatial and nonspatial associations. , 2007, Cerebral cortex.

[17]  Daniel D. Dilks,et al.  The Occipital Place Area Is Causally and Selectively Involved in Scene Perception , 2013, The Journal of Neuroscience.

[18]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[19]  Aude Oliva,et al.  Parametric Coding of the Size and Clutter of Natural Scenes in the Human Brain. , 2014, Cerebral cortex.

[20]  D. Hassabis,et al.  Using Imagination to Understand the Neural Basis of Episodic Memory , 2007, The Journal of Neuroscience.

[21]  I. Toni,et al.  Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex , 2013, Current Biology.

[22]  Jia Liu,et al.  Structural and functional neural correlates of spatial navigation: a combined voxel‐based morphometry and functional connectivity study , 2016, Brain and behavior.

[23]  Dirk B. Walther,et al.  Natural Scene Categories Revealed in Distributed Patterns of Activity in the Human Brain , 2009, The Journal of Neuroscience.

[24]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[25]  Dwight J. Kravitz,et al.  Real-World Scene Representations in High-Level Visual Cortex: It's the Spaces More Than the Places , 2011, The Journal of Neuroscience.

[26]  A. Oliva,et al.  A Real-World Size Organization of Object Responses in Occipitotemporal Cortex , 2012, Neuron.

[27]  Jennifer K. E. Steeves,et al.  Transcranial Magnetic Stimulation to the Transverse Occipital Sulcus Affects Scene but Not Object Processing , 2012, Journal of Cognitive Neuroscience.

[28]  Dwight J. Kravitz,et al.  Deconstructing visual scenes in cortex: gradients of object and spatial layout information. , 2013, Cerebral cortex.

[29]  Stephen C. Strother,et al.  Support vector machines for temporal classification of block design fMRI data , 2005, NeuroImage.

[30]  Pavel Zahorik,et al.  Decoding the direction of auditory motion in blind humans , 2011, NeuroImage.

[31]  Russell A. Epstein.  Parahippocampal and retrosplenial contributions to human spatial navigation , 2008, Trends in Cognitive Sciences.

[32]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[33]  K. Grill-Spector,et al.  Relating retinotopic and object-selective responses in human lateral occipital cortex. , 2008, Journal of neurophysiology.

[34]  M. Goodale,et al.  Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. , 2003, Brain : a journal of neurology.

[35]  Russell A. Epstein,et al.  Constructing scenes from objects in human occipitotemporal cortex , 2011, Nature Neuroscience.

[36]  Irving Biederman,et al.  Object search in nonscene displays. , 1988 .

[37]  Gaspare Galati,et al.  I can see where you would be: Patterns of fMRI activity reveal imagined landmarks , 2017, NeuroImage.

[38]  J. Devlin,et al.  Triple Dissociation of Faces, Bodies, and Objects in Extrastriate Cortex , 2009, Current Biology.

[39]  Yi Chen,et al.  Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): Random permutations and cluster size control , 2011, NeuroImage.

[40]  Guillaume A. Rousselet,et al.  Processing scene context: Fast categorization and object interference , 2007, Vision Research.

[41]  Drew Linsley,et al.  Encoding-Stage Crosstalk Between Object- and Spatial Property-Based Scene Processing Pathways. , 2015, Cerebral cortex.

[42]  Russell A. Epstein,et al.  Neural responses to visual scenes reveals inconsistencies between fMRI adaptation and multivoxel pattern analysis , 2012, Neuropsychologia.

[43]  Alexander G. Huth,et al.  Functional Subdomains within Scene-Selective Cortex: Parahippocampal Place Area, Retrosplenial Complex, and Occipital Place Area , 2016, The Journal of Neuroscience.

[44]  Eleanor A. Maguire,et al.  Retrosplenial Cortex Codes for Permanent Landmarks , 2012, PloS one.

[45]  Rafael Malach,et al.  Large-Scale Mirror-Symmetry Organization of Human Occipito-Temporal Object Areas , 2003, Neuron.

[46]  E. Maguire.  The retrosplenial contribution to human navigation: a review of lesion and neuroimaging findings. , 2001, Scandinavian journal of psychology.

[47]  Bettina Sorger,et al.  Decoding the direction of imagined visual motion using 7 T ultra-high field fMRI , 2016, NeuroImage.

[48]  Jonathan S. Cant,et al.  Feature diagnosticity and task context shape activity in human scene-selective cortex , 2016, NeuroImage.

[49]  Gaoyan Zhang,et al.  Revealing the Semantic Association between Perception of Scenes and Significant Objects by Representational Similarity Analysis , 2017, Neuroscience.

[50]  N. Kanwisher,et al.  Cortical Regions Involved in Perceiving Object Shape , 2000, The Journal of Neuroscience.

[51]  Daniel D. Dilks,et al.  Perceived egocentric distance sensitivity and invariance across scene-selective cortex , 2016, Cortex.

[52]  Yaacov Trope,et al.  Do object-category selective regions in the ventral visual stream represent perceived distance information? , 2012, Brain and Cognition.

[53]  Soojin Park,et al.  Disentangling Scene Content from Spatial Boundary: Complementary Roles for the Parahippocampal Place Area and Lateral Occipital Complex in Representing Real-World Scenes , 2011, The Journal of Neuroscience.

[54]  D. Kumaran,et al.  Double Dissociation between Hippocampal and Parahippocampal Responses to Object–Background Context and Scene Novelty , 2011, The Journal of Neuroscience.

[55]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[56]  Demis Hassabis,et al.  Differential engagement of brain regions within a ‘core’ network during scene construction , 2010, Neuropsychologia.

[57]  Janaina Mourão Miranda,et al.  Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data , 2005, NeuroImage.

[58]  Kazuhiko Yokosawa,et al.  Does object view influence the scene consistency effect? , 2015, Attention, perception & psychophysics.

[59]  S. Edelman,et al.  Cue-Invariant Activation in Object-Related Areas of the Human Occipital Lobe , 1998, Neuron.

[60]  N. Kanwisher,et al.  Mental Imagery of Faces and Places Activates Corresponding Stimulus-Specific Brain Regions , 2000, Journal of Cognitive Neuroscience.

[61]  Daniel D. Dilks,et al.  The occipital place area represents first-person perspective motion information through scenes , 2016, Cortex.

[62]  Michelle R. Greene,et al.  Recognition of natural scenes from global properties: Seeing the forest without representing the trees , 2009, Cognitive Psychology.

[63]  Melvyn A. Goodale,et al.  The role of apparent size in building- and object-specific regions of ventral visual cortex , 2011, Brain Research.

[64]  Sean P. MacEvoy,et al.  Do Simultaneously Viewed Objects Influence Scene Recognition Individually or as Groups? Two Perceptual Studies , 2014, PloS one.

[65]  Daniel D. Dilks,et al.  The occipital place area represents the local elements of scenes , 2016, NeuroImage.

[66]  Daniel D. Dilks,et al.  Mirror-Image Sensitivity and Invariance in Object and Scene Processing Pathways , 2011, The Journal of Neuroscience.

[67]  Jodi L. Davenport,et al.  Scene Consistency in Object and Background Perception , 2004, Psychological science.

[68]  Drew Linsley,et al.  Evidence for participation by object-selective visual cortex in scene category judgments. , 2014, Journal of vision.

[69]  David C. Zhu,et al.  Full Scenes produce more activation than Close-up Scenes and Scene-Diagnostic Objects in parahippocampal and retrosplenial cortex: An fMRI study , 2008, Brain and Cognition.

[70]  Jack L. Gallant,et al.  Fourier power, subjective distance, and object categories all provide plausible models of BOLD responses in scene-selective visual areas , 2015, Front. Comput. Neurosci..

[71]  Russell A. Epstein,et al.  Anchoring the neural compass: Coding of local spatial reference frames in human medial parietal lobe , 2014, Nature Neuroscience.