Visual and Tactile Sensory Systems Share Common Features in Object Recognition

Abstract Although we use our visual and tactile sensory systems interchangeably for object recognition on a daily basis, little is known about the mechanism underlying this ability. This study examined how 3D shape features of objects form two congruent and interchangeable visual and tactile perceptual spaces in healthy male and female participants. Since active exploration plays an important role in shape processing, a virtual reality environment was used to visually explore 3D objects called digital embryos without using the tactile sense. In addition, during the tactile procedure, blindfolded participants actively palpated a 3D-printed version of the same objects with both hands. We first demonstrated that the visual and tactile perceptual spaces were highly similar. We then extracted a series of 3D shape features to investigate how visual and tactile exploration can lead to the correct identification of the relationships between objects. The results indicate that both modalities share the same shape features to form highly similar veridical spaces. This finding suggests that visual and tactile systems might apply similar cognitive processes to sensory inputs that enable humans to rely merely on one modality in the absence of another to recognize surrounding objects.

[1]  Leslie G. Ungerleider,et al.  Curvature processing in human visual cortical areas , 2020, NeuroImage.

[2]  Rustam Stolkin,et al.  Estimation and Exploitation of Objects' Inertial Parameters in Robotic Grasping and Manipulation: A Survey , 2019, Robotics Auton. Syst..

[3]  Knut Drewing,et al.  Memory influences haptic perception of softness , 2019, Scientific Reports.

[4]  Chen Yu,et al.  Active Object Manipulation Facilitates Visual Object Learning: An Egocentric Vision Study , 2019, ArXiv.

[5]  Patrick A Cabe,et al.  Assume a Spherical Chicken: Analytic Constraints, Inertia Tensor Information, and Wielded Rod Length Perception , 2019, Journal of motor behavior.

[6]  K. Drewing,et al.  Top-down modulation of shape and roughness discrimination in active touch by covert attention , 2018, Attention, Perception, & Psychophysics.

[7]  Luis Concha,et al.  Tactile object categories can be decoded from the parietal and lateral-occipital cortices , 2017, Neuroscience.

[8]  Robert A. Jacobs,et al.  From Sensory Signals to Modality-Independent Conceptual Representations: A Probabilistic Language of Thought Approach , 2015, PLoS Comput. Biol..

[9]  C. Wallraven,et al.  Visual and Haptic Shape Processing in the Human Brain: Unisensory Processing, Multisensory Convergence, and Top-Down Influences. , 2015, Cerebral cortex.

[10]  Gideon P. Caplovitz,et al.  The lemon illusion: seeing curvature where there is none , 2015, Front. Hum. Neurosci..

[11]  S. Lacey,et al.  Visuo-haptic multisensory object recognition, categorization, and representation , 2014, Front. Psychol..

[12]  Randall Stilla,et al.  Spatial imagery in haptic shape perception , 2014, Neuropsychologia.

[13]  A. Caramazza,et al.  Nonvisual and Visual Object Shape Representations in Occipitotemporal Cortex: Evidence from Congenitally Blind and Sighted Adults , 2014, The Journal of Neuroscience.

[14]  H. Bülthoff,et al.  The eyes grasp, the hands see: Metric category knowledge transfers between vision and touch , 2013, Psychonomic Bulletin & Review.

[15]  C. Wallraven,et al.  Exploiting object constancy: effects of active exploration and shape morphing on similarity judgments of novel objects , 2013, Experimental Brain Research.

[16]  R. Jacobs,et al.  Transfer of object category knowledge across visual and haptic modalities: Experimental and computational studies , 2013, Cognition.

[17]  Eugene Bart,et al.  Creating Objects and Object Categories for Studying Perception and Perceptual Learning , 2012, Journal of visualized experiments : JoVE.

[18]  Ik Soo Lim,et al.  Curvature and the visual perception of shape: theory on information along object boundaries and the minima rule revisited. , 2012, Psychological review.

[19]  Roberta L Klatzky,et al.  Haptic object perception: spatial dimensionality and relation to vision , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[20]  Astrid M L Kappers,et al.  Human perception of shape from touch , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[21]  H. Bülthoff,et al.  Similarity and categorization: from vision to touch. , 2011, Acta psychologica.

[22]  W. T. Maddox,et al.  Annals of the New York Academy of Sciences Human Category Learning 2.0 Brief Review of First-generation Research , 2022 .

[23]  Susan Haag,et al.  Effects of vision and haptics on categorizing common objects , 2011, Cognitive Processing.

[24]  P. Rossini,et al.  Passive tactile recognition of geometrical shape in humans: An fMRI study , 2010, Brain Research Bulletin.

[25]  H. Bülthoff,et al.  Visual and haptic perceptual spaces show high similarity in humans. , 2010, Journal of vision.

[26]  Christian Wallraven,et al.  Visual and Haptic Perceptual Spaces From Parametrically-Defined to Natural Objects , 2010, AAAI Spring Symposium: Cognitive Shape Processing.

[27]  G. Aguirre,et al.  Different spatial scales of shape similarity representation in lateral and ventral LOC. , 2009, Cerebral cortex.

[28]  Amir Amedi,et al.  A Putative Model of Multisensory Object Representation , 2009, Brain Topography.

[29]  N. Jaworska,et al.  A Review of Multidimensional Scaling (MDS) and its Utility in Various Psychological Domains , 2009 .

[30]  Johan Wagemans,et al.  Perceived Shape Similarity among Unfamiliar Objects and the Organization of the Human Object Vision Pathway , 2008, The Journal of Neuroscience.

[31]  S. Hsiao.  Central mechanisms of tactile shape perception , 2008, Current Opinion in Neurobiology.

[32]  N. Kanwisher,et al.  Multivariate Patterns in Object-Selective Cortex Dissociate Perceptual and Physical Shape Similarity , 2008, PLoS biology.

[33]  Hideko F. Norman,et al.  Learning to Perceive Differences in Solid Shape through Vision and Touch , 2008, Perception.

[34]  H. Bülthoff,et al.  Multimodal similarity and categorization of novel, three-dimensional objects , 2007, Neuropsychologia.

[35]  Heinrich H. Bülthoff,et al.  Analyzing Perceptual Representations of Complex, Parametrically-Defined Shapes Using MDS , 2007, EuroHaptics.

[36]  Heinrich H. Bülthoff,et al.  Object Feature Validation Using Visual and Haptic Similarity Ratings , 2022 .

[37]  Roberto Casati,et al.  Is the subjective feel of "presence" an uninteresting goal? , 2005, J. Vis. Lang. Comput..

[38]  Florian Steinke,et al.  A similarity-based approach to perceptual feature validation , 2005, APGV '05.

[39]  Heinrich H. Bülthoff,et al.  A comparison of visual and haptic object representations based on similarity , 2005, Ninth International Conference on Information Visualisation (IV'05).

[40]  Szymon Rusinkiewicz,et al.  Estimating curvatures and their derivatives on triangle meshes , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[41]  I. Gauthier,et al.  Computational approaches to the development of perceptual expertise , 2004, Trends in Cognitive Sciences.

[42]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[43]  J. Todd.  Review TRENDS in Cognitive Sciences Vol.8 No.3 March 2004 The visual perception of 3D shape q , 2022 .

[44]  Hideko F. Norman,et al.  The visual and haptic perception of natural object shape , 2004, Perception & Psychophysics.

[45]  Daniel Kersten,et al.  Bootstrapped learning of novel objects. , 2003, Journal of vision.

[46]  T. Hendler,et al.  Convergence of visual and tactile shape processing in the human lateral occipital complex. , 2002, Cerebral cortex.

[47]  T Vilis,et al.  “Active” and “passive” learning of three-dimensional object structure within an immersive virtual reality environment , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[48]  Michel Vidal-Naquet,et al.  Visual features of intermediate complexity and their use in classification , 2002, Nature Neuroscience.

[49]  R. Shepard.  Perceptual-cognitive universals as reflections of the world. , 2001, The Behavioral and brain sciences.

[50]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[51]  H. Bülthoff,et al.  Viewpoint Dependence in Visual and Haptic Object Recognition , 2001, Psychological science.

[52]  M. Goodale,et al.  Active manual control of object views facilitates visual recognition , 1999, Current Biology.

[53]  Heinrich H. Bülthoff,et al.  Navigating through a virtual city: Using virtual reality technology to study human action and perception , 1998, Future Gener. Comput. Syst..

[54]  J J Koenderink,et al.  Haptic curvature discrimination at several regions of the hand , 1997, Perception & psychophysics.

[55]  C Carello,et al.  Selective perception by dynamic touch , 1996, Perception & psychophysics.

[56]  Martial Hebert,et al.  On 3D shape similarity , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[57]  M. Turvey,et al.  Eigenvalues of the inertia tensor and exteroception by the “muscular sense” , 1994, Neuroscience.

[58]  M T Turvey,et al.  Role of the inertia tensor in haptically perceiving where an object is grasped. , 1994, Journal of experimental psychology. Human perception and performance.

[59]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[61]  R L Klatzky,et al.  Identifying objects by touch: An “expert system” , 1985, Perception & psychophysics.

[62]  Michael C. Hout,et al.  Multidimensional Scaling , 2003, Encyclopedic Dictionary of Archaeology.

[63]  C. Wallraven,et al.  Categorizing natural objects: a comparison of the visual and the haptic modalities , 2011, Experimental Brain Research.

[64]  Paolo Cignoni,et al.  MeshLab: an Open-Source Mesh Processing Tool , 2008, Eurographics Italian Chapter Conference.

[65]  Johan Wagemans,et al.  The effect of category learning on the representation of shape: dimensions can be biased but not differentiated. , 2003, Journal of experimental psychology. General.

[66]  James T Todd,et al.  The visual perception of 3-D shape from multiple cues: Are observers capable of perceiving metric structure? , 2003, Perception & psychophysics.

[67]  I. Kingma,et al.  The Inertia Tensor Versus Static Moment and Mass in Perceiving Length and Heaviness of Hand-Wielded Rods , 2002 .

[68]  K. R. Clarke,et al.  Change in marine communities : an approach to statistical analysis and interpretation , 2001 .

[69]  James A. Hampton,et al.  Similarity and Categorization , 2001 .

[70]  Shimon Edelman,et al.  Representation and recognition in vision , 1999 .

[71]  G. Krieger,et al.  Curvature Measures in Visual Information Processing , 1998 .

[72]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .