Representing stuff in the human brain

Our experience of materials does not merely comprise judgments of single properties such as glossiness or roughness but is rather made up of a multitude of simultaneous impressions of qualities. To understand the neural mechanisms yielding such complex impressions, we suggest that it is necessary to extend existing experimental approaches to those that view material perception as a distributed and dynamic process. A distributed representations framework not only fits better with our perceptual experience of material qualities, it is commensurate with recent psychophysics and neuroimaging results.

[1]  Melvyn A. Goodale,et al.  Category-specific neural processing for naming pictures of animals and naming pictures of tools: An ALE meta-analysis , 2010, Neuropsychologia.

[2]  Dwight J. Kravitz,et al.  The ventral visual pathway: an expanded neural framework for the processing of object quality , 2013, Trends in Cognitive Sciences.

[3]  Hanhan Wei,et al.  Can you see what you feel? Color and folding properties affect visual-tactile material discrimination of fabrics. , 2016, Journal of vision.

[4]  Robert W. Kentridge,et al.  Separate processing of texture and form in the ventral stream: evidence from FMRI and visual agnosia. , 2010, Cerebral cortex.

[5]  Seong-Whan Lee,et al.  Human visual cortical responses to specular and matte motion flows , 2015, Front. Hum. Neurosci..

[6]  Guy A. Orban,et al.  The Extraction of 3D Shape from Texture and Shading in the Human Brain , 2008, Cerebral cortex.

[7]  Erich W. Graf,et al.  Touch influences perceived gloss , 2016, Scientific Reports.

[8]  W. Bair,et al.  Neural Coding for Shape and Texture in Macaque Area V4 , 2019, The Journal of Neuroscience.

[9]  Karl R. Gegenfurtner,et al.  The representation of material categories in the brain , 2014, Front. Psychol..

[10]  Robert W. Kentridge,et al.  Separate channels for processing form, texture, and color: evidence from FMRI adaptation and visual object agnosia. , 2010, Cerebral cortex.

[11]  Naokazu Goda,et al.  Neural mechanisms of material perception: quest on Shitsukan , 2018, Neuroscience.

[12]  Jonathan S. Cant,et al.  Cerebral Cortex Advance Access published April 28, 2006 Attention to Form or Surface Properties Modulates Different Regions of Human , 2022 .

[13]  Randall Stilla,et al.  Selective visuo‐haptic processing of shape and texture , 2008, Human brain mapping.

[14]  R. W. Kentridge,et al.  The perception of gloss: A review , 2015, Vision Research.

[15]  Christiane B Wiebel,et al.  Visual and haptic representations of material properties. , 2013, Multisensory research.

[16]  Phillip J. Marlow,et al.  Photogeometric Cues to Perceived Surface Shading , 2019, Current Biology.

[17]  Barton L. Anderson,et al.  Coupled computations of three-dimensional shape and material , 2015, Current Biology.

[18]  Barton L Anderson,et al.  Perception and misperception of surface opacity , 2017, Proceedings of the National Academy of Sciences.

[19]  Nina S. Hsu,et al.  Chromaticity of color perception and object color knowledge , 2012, Neuropsychologia.

[20]  Midori Tokita,et al.  Perception of the material properties of wood based on vision, audition, and touch , 2015, Vision Research.

[21]  Alex Martin.  GRAPES—Grounding representations in action, perception, and emotion systems: How object properties and categories are represented in the human brain , 2015, Psychonomic Bulletin & Review.

[22]  Jonathan S. Cant,et al.  fMR-adaptation reveals separate processing regions for the perception of form and texture in the human ventral stream , 2008, Experimental Brain Research.

[23]  Barton L Anderson,et al.  Visual perception of materials and surfaces , 2011, Current Biology.

[24]  Chris I. Baker,et al.  Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images , 2018, NeuroImage.

[25]  Alexandra C. Schmid,et al.  Perceptual dimensions underlying lightness perception in homogeneous center-surround displays. , 2017, Journal of vision.

[26]  John G. Mikhael,et al.  Functional neuroanatomy of intuitive physical inference , 2016, Proceedings of the National Academy of Sciences.

[27]  Dwight J. Kravitz,et al.  Task context impacts visual object processing differentially across the cortex , 2014, Proceedings of the National Academy of Sciences.

[28]  R VanRullen,et al.  Is it a Bird? Is it a Plane? Ultra-Rapid Visual Categorisation of Natural and Artifactual Objects , 2001, Perception.

[29]  Katja Doerschner,et al.  Shatter and splatter: The contribution of mechanical and optical properties to the perception of soft and hard breaking materials. , 2018, Journal of vision.

[30]  Bradford Z. Mahon,et al.  What is embodied about cognition? , 2015, Language, cognition and neuroscience.

[31]  Knut Drewing,et al.  Systematic Relations between Affective and Sensory Material Dimensions in Touch , 2018, IEEE Transactions on Haptics.

[32]  E. Adelson,et al.  Accuracy and speed of material categorization in real-world images. , 2014, Journal of vision.

[33]  Radoslaw Martin Cichy,et al.  The representational dynamics of task and object processing in humans , 2018, eLife.

[34]  Dimitrios Pantazis,et al.  Similarity-based fusion of MEG and fMRI reveals spatio-temporal dynamics in human cortex during visual object recognition , 2015 .

[35]  Dwight J. Kravitz,et al.  A new neural framework for visuospatial processing , 2011, Nature Reviews Neuroscience.

[36]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[37]  K. Gegenfurtner,et al.  Image Statistics and the Representation of Material Properties in the Visual Cortex , 2016, Front. Psychol..

[38]  A. Parker,et al.  Perceptually Bistable Three-Dimensional Figures Evoke High Choice Probabilities in Cortical Area MT , 2001, The Journal of Neuroscience.

[39]  Chris I. Baker,et al.  Deconstructing multivariate decoding for the study of brain function , 2017, NeuroImage.

[40]  Christiane B. Wiebel,et al.  Perceptual qualities and material classes. , 2013, Journal of vision.

[41]  Shin'ya Nishida,et al.  Audiovisual integration in the human perception of materials. , 2014, Journal of vision.

[42]  A. Amedi,et al.  Functional imaging of human crossmodal identification and object recognition , 2005, Experimental Brain Research.

[43]  Kingson Man,et al.  Convergent and invariant object representations for sight, sound, and touch , 2015, Human brain mapping.

[44]  Massimiliano Di Luca,et al.  Differential processing of binocular and monocular gloss cues in human visual cortex , 2016, Journal of neurophysiology.

[45]  B. Anderson,et al.  Specular Image Structure Modulates the Perception of Three-Dimensional Shape , 2014, Current Biology.

[46]  Andrew E Welchman,et al.  The Human Brain in Depth: How We See in 3D. , 2016, Annual review of vision science.

[47]  Akichika Mikami,et al.  Neuronal correlates of motion‐defined shape perception in primate dorsal and ventral streams , 2018, The European journal of neuroscience.

[48]  J Brendan Ritchie,et al.  Decoding the Brain: Neural Representation and the Limits of Multivariate Pattern Analysis in Cognitive Neuroscience , 2017, bioRxiv.

[49]  Roland W Fleming,et al.  Influence of optical material properties on the perception of liquids. , 2016, Journal of vision.

[50]  Xiaoping Hu,et al.  Dual pathways for haptic and visual perception of spatial and texture information , 2011, NeuroImage.

[51]  Jonathan S. Cant,et al.  Scratching Beneath the Surface: New Insights into the Functional Properties of the Lateral Occipital Area and Parahippocampal Place Area , 2011, The Journal of Neuroscience.

[52]  Roland W Fleming,et al.  Material Perception. , 2017, Annual review of vision science.

[53]  Hiroshi Ban,et al.  fMRI evidence for areas that process surface gloss in the human visual cortex , 2015, Vision Research.

[54]  M. Landy,et al.  Measurement and modeling of depth cue combination: in defense of weak fusion , 1995, Vision Research.

[55]  G. Orban,et al.  Attention to 3-D Shape, 3-D Motion, and Texture in 3-D Structure from Motion Displays , 2004, Journal of Cognitive Neuroscience.

[56]  Katja Doerschner,et al.  Chocolate, chrome, or cloth? The appearance of specular highlights determines perceived material category , 2019 .

[57]  Dorita H. F. Chang,et al.  Look but don't touch: Visual cues to surface structure drive somatosensory cortex , 2016, NeuroImage.

[58]  Jonathan S. Cant,et al.  Object Ensemble Processing in Human Anterior-Medial Ventral Visual Cortex , 2012, The Journal of Neuroscience.

[59]  Naokazu Goda,et al.  Transformation from image-based to perceptual representation of materials along the human ventral visual pathway , 2011, NeuroImage.

[60]  Carlos Aliaga,et al.  Sackcloth or silk?: the impact of appearance vs dynamics on the perception of animated cloth , 2015, SAP.

[61]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[62]  Li Fei-Fei,et al.  Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior , 2018, eLife.

[63]  D. Knill,et al.  Apparent surface curvature affects lightness perception , 1991, Nature.

[64]  H. P. Op de Beeck,et al.  Task Context Overrules Object- and Category-Related Representational Content in the Human Parietal Cortex , 2017, Cerebral cortex.

[65]  Barton L Anderson,et al.  Motion and texture shape cues modulate perceived material properties. , 2016, Journal of vision.

[66]  Alex Martin,et al.  Spontaneous resting-state BOLD fluctuations reveal persistent domain-specific neural networks. , 2012, Social cognitive and affective neuroscience.

[67]  Jonathan S. Cant,et al.  The Contribution of Object Shape and Surface Properties to Object Ensemble Representation in Anterior-medial Ventral Visual Cortex , 2017, Journal of Cognitive Neuroscience.

[68]  Hiroshi Ando,et al.  Human cortical areas involved in perception of surface glossiness , 2014, NeuroImage.

[69]  Alfonso Caramazza,et al.  Unconscious processing dissociates along categorical lines , 2008, Proceedings of the National Academy of Sciences.

[70]  Barton L. Anderson,et al.  Material properties derived from three-dimensional shape representations , 2015, Vision Research.

[71]  S. Lacey,et al.  Visuo-haptic multisensory object recognition, categorization, and representation , 2014, Front. Psychol..

[72]  Dinesh K. Pai,et al.  Perception of Material from Contact Sounds , 2000, Presence: Teleoperators & Virtual Environments.

[73]  Jonathan S. Cant,et al.  Independent Processing of Form, Colour, and Texture in Object Perception , 2008, Perception.

[74]  G. Orban,et al.  Perception of Three-Dimensional Shape From Specular Highlights, Deformations of Shading, and Other Types of Visual Information , 2004, Psychological science.

[75]  S. Thorpe,et al.  Speed of processing in the human visual system , 1996, Nature.

[76]  Hanna Damasio,et al.  Predicting visual stimuli on the basis of activity in auditory cortices , 2010, Nature Neuroscience.

[77]  Fraser W. Smith,et al.  Decoding Visual Object Categories in Early Somatosensory Cortex , 2013, Cerebral cortex.

[78]  Melvyn A. Goodale,et al.  Are visual texture-selective areas recruited during haptic texture discrimination? , 2013, NeuroImage.

[79]  H. Bülthoff,et al.  3D shape perception from combined depth cues in human visual cortex , 2005, Nature Neuroscience.

[80]  Chris I Baker,et al.  Contributions of low- and high-level properties to neural processing of visual scenes in the human brain , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[81]  Jonathan S. Cant,et al.  The Impact of Density and Ratio on Object-Ensemble Representation in Human Anterior-Medial Ventral Visual Cortex. , 2015, Cerebral cortex.