Parsing rooms: the role of the PPA and RSC in perceiving object relations and spatial layout

The perception of a scene involves grasping the global space of the scene, usually called the spatial layout, as well as the objects in the scene and the relations between them. The main brain areas involved in scene perception, the parahippocampal place area (PPA) and retrosplenial cortex (RSC), are supposed to mostly support the processing of spatial layout. Here we manipulated the objects and their relations either by arranging objects within rooms in a common way or by scattering them randomly. The rooms were then varied for spatial layout by keeping or removing the walls of the room, a typical layout manipulation. We then combined a visual search paradigm, where participants actively search for an object within the room, with multivariate pattern analysis (MVPA). Both left and right PPA were sensitive to the layout properties, but the right PPA was also sensitive to the object relations even when the information about objects and their relations is used in the cross-categorization procedure on novel stimuli. The left and right RSC were sensitive to both spatial layout and object relations, but could only use the information about object relations for cross-categorization to novel stimuli. These effects were restricted to the PPA and RSC, as other control brain areas did not display the same pattern of results. Our results underline the importance of employing paradigms that require participants to explicitly retrieve domain-specific processes and indicate that objects and their relations are processed in the scene areas to a larger extent than previously assumed.

[1]  Seth E. Bouvier,et al.  Temporal Components in the Parahippocampal Place Area Revealed by Human Intracerebral Recordings , 2013, The Journal of Neuroscience.

[2]  Daniel D. Dilks,et al.  The occipital place area represents the local elements of scenes , 2016, NeuroImage.

[3]  Jennifer J. Richler,et al.  Meanings, Mechanisms, and Measures of Holistic Processing , 2012, Front. Psychology.

[4]  Yaacov Trope,et al.  Do object-category selective regions in the ventral visual stream represent perceived distance information? , 2012, Brain and Cognition.

[5]  Soojin Park,et al.  Disentangling Scene Content from Spatial Boundary: Complementary Roles for the Parahippocampal Place Area and Lateral Occipital Complex in Representing Real-World Scenes , 2011, The Journal of Neuroscience.

[6]  R. Malach,et al.  Top-down engagement modulates the neural expressions of visual expertise. , 2010, Cerebral cortex.

[7]  Dirk B. Walther,et al.  Natural Scene Categories Revealed in Distributed Patterns of Activity in the Human Brain , 2009, The Journal of Neuroscience.

[8]  Paul E. Downing,et al.  Viewpoint-Specific Scene Representations in Human Parahippocampal Cortex , 2003, Neuron.

[9]  Daniel C. Krawczyk,et al.  The neural organization of perception in chess experts , 2011, Neuroscience Letters.

[10]  Russell A. Epstein,et al.  The Parahippocampal Place Area Recognition, Navigation, or Encoding? , 1999, Neuron.

[11]  Rankin W. McGugin,et al.  High-resolution imaging of expertise reveals reliable object selectivity in the fusiform face area related to perceptual performance , 2012, Proceedings of the National Academy of Sciences.

[12]  Moshe Bar,et al.  The resilience of object predictions: Early recognition across viewpoints and exemplars , 2014, Psychonomic bulletin & review.

[13]  J. Wolfe,et al.  Differential Electrophysiological Signatures of Semantic and Syntactic Scene Processing , 2013, Psychological science.

[14]  M Erb,et al.  Differential activation in parahippocampal and prefrontal cortex during word and face encoding tasks , 2001, Neuroreport.

[15]  M. Bar Visual objects in context , 2004, Nature Reviews Neuroscience.

[16]  I. Biederman,et al.  On the information extracted from a glance at a scene. , 1974, Journal of experimental psychology.

[17]  Russell A. Epstein,et al.  Constructing scenes from objects in human occipitotemporal cortex , 2011, Nature Neuroscience.

[18]  N. Kanwisher,et al.  Domain specificity in visual cortex. , 2006, Cerebral cortex.

[19]  Eleanor A Maguire,et al.  A New Role for the Parahippocampal Cortex in Representing Space , 2011, The Journal of Neuroscience.

[20]  Done-Sik Yoo,et al.  Neural correlates related to action observation in expert archers , 2011, Behavioural Brain Research.

[21]  K J Vicente,et al.  An ecological theory of expertise effects in memory recall. , 1998, Psychological review.

[22]  Michelle R. Greene,et al.  Visual search in scenes involves selective and nonselective pathways , 2011, Trends in Cognitive Sciences.

[23]  Jodi L. Davenport,et al.  Scene Consistency in Object and Background Perception , 2004, Psychological science.

[24]  Michael Erb,et al.  Mechanisms and neural basis of object and pattern recognition: a study with chess experts. , 2010, Journal of experimental psychology. General.

[25]  M. Bar,et al.  The role of the parahippocampal cortex in cognition , 2013, Trends in Cognitive Sciences.

[26]  M. Fabre-Thorpe,et al.  A need for more information uptake but not focused attention to access basic-level representations. , 2012, Journal of vision.

[27]  Assaf Harel,et al.  A Neurocognitive Approach to Expertise in Visual Object Recognition , 2015, HCI.

[28]  Brad Wyble,et al.  Detecting meaning in RSVP at 13 ms per picture , 2013, Attention, perception & psychophysics.

[29]  Jim M. Monti,et al.  Neural repetition suppression reflects fulfilled perceptual expectations , 2008, Nature Neuroscience.

[30]  G. Rees,et al.  Fine-scale activity patterns in high-level visual areas encode the category of invisible objects. , 2008, Journal of vision.

[31]  Dwight J. Kravitz,et al.  Real-World Scene Representations in High-Level Visual Cortex: It's the Spaces More Than the Places , 2011, The Journal of Neuroscience.

[32]  Endel Tulving,et al.  The effect of alphabetical subjective organization on memorizing unrelated words. , 1962 .

[33]  Michael Esterman,et al.  Perceptual expectation evokes category-selective cortical activity. , 2010, Cerebral cortex.

[34]  Melvyn A. Goodale,et al.  The role of apparent size in building- and object-specific regions of ventral visual cortex , 2011, Brain Research.

[35]  R. Yin Looking at Upside-down Faces , 1969 .

[36]  Merim Bilalic,et al.  The Faces in Radiological Images: Fusiform Face Area Supports Radiological Expertise. , 2016, Cerebral cortex.

[37]  Jonathan D. Cohen,et al.  Confounds in multivariate pattern analysis: Theory and rule representation case study , 2013, NeuroImage.

[38]  M. Bar,et al.  The parahippocampal cortex mediates spatial and nonspatial associations. , 2007, Cerebral cortex.

[39]  Fernand Gobet,et al.  Structure and Stimulus Familiarity: A Study of Memory in Chess-Players with Functional Magnetic Resonance Imaging , 2005, The Spanish Journal of Psychology.

[40]  C. Summerfield,et al.  Expectation (and attention) in visual cognition , 2009, Trends in Cognitive Sciences.

[41]  J M Mandler,et al.  Some of the thousand words a picture is worth. , 1976, Journal of experimental psychology. Human learning and memory.

[42]  J. S. Guntupalli,et al.  Decoding neural representational spaces using multivariate pattern analysis. , 2014, Annual review of neuroscience.

[43]  Kalanit Grill-Spector,et al.  Representation of shapes, edges, and surfaces across multiple cues in the human visual cortex. , 2008, Journal of neurophysiology.

[44]  Itamar Kahn,et al.  Hemispheric asymmetry of visual scene processing in the human brain: evidence from repetition priming and intrinsic activity. , 2012, Cerebral cortex.

[45]  Li Fei-Fei,et al.  Two Distinct Scene-Processing Networks Connecting Vision and Memory , 2016, eNeuro.

[46]  Fei Guo,et al.  Neural Representations of Contextual Guidance in Visual Search of Real-World Scenes , 2013, The Journal of Neuroscience.

[47]  N. Kanwisher,et al.  The fusiform face area: a cortical region specialized for the perception of faces , 2006, Philosophical Transactions of the Royal Society B: Biological Sciences.

[48]  W. Grodd,et al.  Many Faces of Expertise: Fusiform Face Area in Chess Experts and Novices , 2011, The Journal of Neuroscience.

[49]  Michael J. Tarr,et al.  Associative Processing Is Inherent in Scene Perception , 2015, PloS one.

[50]  M. Bar,et al.  Scenes Unseen: The Parahippocampal Cortex Intrinsically Subserves Contextual Associations, Not Scenes or Places Per Se , 2008, The Journal of Neuroscience.

[51]  S. Edelman,et al.  Differential Processing of Objects under Various Viewing Conditions in the Human Lateral Occipital Complex , 1999, Neuron.

[52]  Russell A. Epstein,et al.  Two kinds of FMRI repetition suppression? Evidence for dissociable neural mechanisms. , 2008, Journal of neurophysiology.

[53]  Russell A. Epstein,et al.  Coding of navigational affordances in the human visual system , 2017, Proceedings of the National Academy of Sciences.

[54]  T. Kimura,et al.  Mental navigation in humans is processed in the anterior bank of the parieto-occipital sulcus , 2002, Neuroscience Letters.

[55]  A M Dale,et al.  Optimal experimental design for event‐related fMRI , 1999, Human brain mapping.

[56]  A. Oliva,et al.  A Real-World Size Organization of Object Responses in Occipitotemporal Cortex , 2012, Neuron.

[57]  Jonathan S. Cant,et al.  Feature diagnosticity and task context shape activity in human scene-selective cortex , 2016, NeuroImage.

[58]  Aude Oliva,et al.  Parametric Coding of the Size and Clutter of Natural Scenes in the Human Brain. , 2014, Cerebral cortex.

[59]  J. Wolfe,et al.  The interplay of episodic and semantic memory in guiding repeated search in scenes , 2013, Cognition.

[60]  M. Farah,et al.  Parts and Wholes in Face Recognition , 1993, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[61]  G. Ritchey,et al.  Long-Term Memory for Pictures , 2005 .

[62]  Natalie E. Hansen,et al.  The influence of behavioral relevance on the processing of global scene properties: An ERP study , 2018, Neuropsychologia.

[63]  M. Tarr,et al.  Activation of the middle fusiform 'face area' increases with expertise in recognizing novel objects , 1999, Nature Neuroscience.

[64]  Dwight J. Kravitz,et al.  Task context impacts visual object processing differentially across the cortex , 2014, Proceedings of the National Academy of Sciences.

[65]  N. Gronau,et al.  The Necessity of Visual Attention to Scene Categorization: Dissociating “Task-Relevant” and “Task-Irrelevant” Scene Distractors , 2017, Journal of experimental psychology. Human perception and performance.

[66]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[67]  Li Fei-Fei,et al.  Differential Connectivity Within the Parahippocampal Place Area , 2013 .

[68]  Martin N. Hebart,et al.  The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data , 2015, Front. Neuroinform..

[69]  K. Amunts,et al.  Brodmann's Areas 17 and 18 Brought into Stereotaxic Space—Where and How Variable? , 2000, NeuroImage.

[70]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[71]  Dwight J. Kravitz,et al.  A new neural framework for visuospatial processing , 2011, Nature Reviews Neuroscience.

[72]  I. Biederman,et al.  Searching for objects in real-world scences. , 1973, Journal of experimental psychology.

[73]  K. A. Ericsson,et al.  Expert and exceptional performance: evidence of maximal adaptation to task constraints. , 1996, Annual review of psychology.

[74]  I. Biederman,et al.  Scene perception: Detecting and judging objects undergoing relational violations , 1982, Cognitive Psychology.

[75]  Michael Erb,et al.  Expertise modulates the neural basis of context dependent recognition of objects and their relations , 2012, Human brain mapping.

[76]  Michelle R. Greene,et al.  PSYCHOLOGICAL SCIENCE Research Article The Briefest of Glances The Time Course of Natural Scene Understanding , 2022 .

[77]  J. Haynes Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .

[78]  Daniel C. Krawczyk,et al.  Expertise and processing distorted structure in chess , 2013, Front. Hum. Neurosci..

[79]  M. Brodeur,et al.  The Bank of Standardized Stimuli (BOSS), a New Set of 480 Normative Photos of Objects to Be Used as Visual Stimuli in Cognitive Research , 2010, PloS one.

[80]  Olivia S. Cheung,et al.  Visual prediction and perceptual expertise. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[81]  Isabel Gauthier,et al.  Revisiting the role of spatial frequencies in the holistic processing of faces. , 2008, Journal of experimental psychology. Human perception and performance.

[82]  I. Gauthier,et al.  Expertise for cars and birds recruits brain areas involved in face recognition , 2000, Nature Neuroscience.

[83]  H. Simon,et al.  Perception in chess , 1973 .

[84]  George A. Alvarez,et al.  Natural-Scene Perception Requires Attention , 2011, Psychological science.

[85]  Mary C Potter,et al.  Detecting and remembering simultaneous pictures in a rapid serial visual presentation. , 2009, Journal of experimental psychology. Human perception and performance.

[86]  George L. Malcolm,et al.  Making Sense of Real-World Scenes , 2016, Trends in Cognitive Sciences.

[87]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[88]  Robert Desimone,et al.  Cortical Connections of Area V4 in the Macaque , 2008 .

[89]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[90]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[91]  Jodi L. Davenport Consistency effects between objects in scenes , 2007, Memory & cognition.

[92]  Yetta Kwailing Wong,et al.  Perceptual Expertise and Top–Down Expectation of Musical Notation Engages the Primary Visual Cortex , 2014, Journal of Cognitive Neuroscience.

[93]  Dwight J. Kravitz,et al.  Deconstructing visual scenes in cortex: gradients of object and spatial layout information. , 2013, Cerebral cortex.

[94]  Endel Tulving,et al.  Ecphoric processes in episodic memory , 1983 .

[95]  John D. E. Gabrieli,et al.  Human Neuroscience , 2022 .

[96]  Russell A. Epstein Parahippocampal and retrosplenial contributions to human spatial navigation , 2008, Trends in Cognitive Sciences.

[97]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[98]  Russell A. Epstein,et al.  Visual scene processing in familiar and unfamiliar environments. , 2007, Journal of neurophysiology.

[99]  I Biederman,et al.  Do Background Depth Gradients Facilitate Object Identification? , 1981, Perception.

[100]  Fernand Gobet,et al.  BRAIN LOCALIZATION OF MEMORY CHUNKS IN CHESSPLAYERS , 2007, The International journal of neuroscience.

[101]  R. Fisher 014: On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample. , 1921 .

[102]  Robert Desimone,et al.  Cortical connections of area V4 in the macaque. , 2000, Cerebral cortex.

[103]  Olivier R. Joubert,et al.  How long to get to the “gist” of real-world natural scenes? , 2005 .

[104]  Russell A. Epstein,et al.  Multiple object properties drive scene-selective regions. , 2014, Cerebral cortex.

[105]  Keiji Tanaka,et al.  The Neural Basis of Intuitive Best Next-Move Generation in Board Game Experts , 2011, Science.

[106]  Rankin W. McGugin,et al.  Expertise Effects in Face-Selective Areas are Robust to Clutter and Diverted Attention, but not to Competition. , 2015, Cerebral cortex.

[107]  Fei-Fei Li,et al.  Differential connectivity within the Parahippocampal Place Area , 2013, NeuroImage.

[108]  Richard S. J. Frackowiak,et al.  Recalling Routes around London: Activation of the Right Hippocampus in Taxi Drivers , 1997, The Journal of Neuroscience.

[109]  Emily J. Ward,et al.  How reliable are visual context effects in the parahippocampal place area? , 2010, Cerebral cortex.

[110]  Merim Bilalić,et al.  The Neuroscience of Expertise , 2017 .

[111]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[112]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[113]  Assaf Harel,et al.  What is special about expertise? Visual expertise reveals the interactive nature of real-world object recognition , 2016, Neuropsychologia.

[114]  Russell A. Epstein,et al.  Outside Looking In: Landmark Generalization in the Human Navigational System , 2015, The Journal of Neuroscience.

[115]  R. E. Parker,et al.  Memory for descriptive and spatial information in complex pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[116]  John M. Henderson,et al.  Cortical activation to indoor versus outdoor scenes: an fMRI study , 2007, Experimental Brain Research.

[117]  Rankin W. McGugin,et al.  Robust expertise effects in right FFA , 2014, Neuropsychologia.

[118]  Yaoda Xu,et al.  The Role of Transverse Occipital Sulcus in Scene Perception and Its Relationship to Object Individuation in Inferior Intraparietal Sulcus , 2013, Journal of Cognitive Neuroscience.

[119]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[120]  Merim Bilalic,et al.  Revisiting the Role of the Fusiform Face Area in Expertise , 2016, Journal of Cognitive Neuroscience.

[121]  Li Fei-Fei,et al.  Simple line drawings suffice for functional MRI decoding of natural scene categories , 2011, Proceedings of the National Academy of Sciences.