Vision holds a greater share in visuo-haptic object recognition than touch

The integration of visual and haptic input can facilitate object recognition. Yet, vision might dominate visuo-haptic interactions as it is more effective than haptics in processing several object features in parallel and recognizing objects outside of reaching space. The maximum likelihood approach of multisensory integration would predict that haptics as the less efficient sense for object recognition gains more from integrating additional visual information than vice versa. To test for asymmetries between vision and touch in visuo-haptic interactions, we measured regional changes in brain activity using functional magnetic resonance imaging while healthy individuals performed a delayed-match-to-sample task. We manipulated identity matching of sample and target objects: We hypothesized that only coherent visual and haptic object features would activate unified object representations. The bilateral object-specific lateral occipital cortex, fusiform gyrus, and intraparietal sulcus showed increased activation to crossmodal compared to unimodal matching but only for congruent object pairs. Critically, the visuo-haptic interaction effects in these regions depended on the sensory modality which processed the target object, being more pronounced for haptic than visual targets. This preferential response of visuo-haptic regions indicates a modality-specific asymmetry in crossmodal matching of visual and haptic object features, suggesting a functional primacy of vision over touch in visuo-haptic object recognition.

[1]  G. Kreiman,et al.  Timing, Timing, Timing: Fast Decoding of Object Information from Intracranial Field Potentials in Human Visual Cortex , 2009, Neuron.

[2]  K. Zilles,et al.  Crossmodal Processing of Object Features in Human Anterior Intraparietal Cortex An fMRI Study Implies Equivalencies between Humans and Monkeys , 2002, Neuron.

[3]  Hartwig R. Siebner,et al.  The left fusiform gyrus hosts trisensory representations of manipulable objects , 2011, NeuroImage.

[4]  S. Thorpe,et al.  Speed of processing in the human visual system , 1996, Nature.

[5]  J L Lancaster,et al.  Automated Talairach Atlas labels for functional brain mapping , 2000, Human brain mapping.

[6]  Xiaoping Hu,et al.  Object familiarity modulates effective connectivity during haptic shape perception , 2010, NeuroImage.

[7]  T. Hendler,et al.  Convergence of visual and tactile shape processing in the human lateral occipital complex. , 2002, Cerebral cortex.

[8]  Timothy Edward John Behrens,et al.  Non-invasive mapping of connections between human thalamus and cortex using diffusion imaging , 2003, Nature Neuroscience.

[9]  R. Goebel,et al.  Multisensory functional magnetic resonance imaging: a future perspective , 2009, Experimental Brain Research.

[10]  T. James,et al.  An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI , 2009, Experimental Brain Research.

[11]  Simon Lacey,et al.  Object familiarity modulates the relationship between visual object imagery and haptic shape perception , 2010, NeuroImage.

[12]  K. Amunts,et al.  The human parietal operculum. II. Stereotaxic maps and correlation with functional imaging results. , 2006, Cerebral cortex.

[14]  Timothy Edward John Behrens,et al.  Functional-anatomical validation and individual variation of diffusion tractography-based segmentation of the human thalamus. , 2005, Cerebral cortex.

[15]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[16]  Randall Stilla,et al.  Selective visuo‐haptic processing of shape and texture , 2008, Human brain mapping.

[17]  Michael J. Martinez,et al.  Bias between MNI and Talairach coordinates analyzed using the ICBM‐152 brain template , 2007, Human brain mapping.

[18]  Andrew Simpson,et al.  Subitizing in Tactile Perception , 2006, Psychological science.

[19]  J. Aggleton,et al.  The functional anatomy of visual-tactile integration in man: a study using positron emission tomography , 2000, Neuropsychologia.

[20]  M. Ernst,et al.  Optimal integration of shape information from vision and touch , 2007, Experimental Brain Research.

[21]  A. Schleicher,et al.  The human parietal operculum. I. Cytoarchitectonic mapping of subdivisions. , 2006, Cerebral cortex.

[22]  K Sathian,et al.  Multisensory object representation: insights from studies of vision and touch. , 2011, Progress in brain research.

[23]  K. Grill-Spector,et al.  Repetition and the brain: neural models of stimulus-specific effects , 2006, Trends in Cognitive Sciences.

[24]  Amir Amedi,et al.  Multisensory visual–tactile object related network in humans: insights gained using a novel crossmodal adaptation approach , 2009, Experimental Brain Research.

[25]  Jochen Kaiser,et al.  Multisensory Functional Magnetic Resonance Imaging , 2010 .

[26]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[27]  Mareike M. Menz,et al.  Multisensory interactions between auditory and haptic object recognition. , 2013, Cerebral cortex.

[28]  齋藤 大輔,et al.  Tactile-visual cross-modal shape matching : A functional MRI study , 2003 .

[29]  Jochen Kaiser,et al.  Audiovisual Functional Magnetic Resonance Imaging Adaptation Reveals Multisensory Integration Effects in Object-Related Sensory Cortices , 2010, The Journal of Neuroscience.

[30]  M. Annett A classification of hand preference by association analysis. , 1970, British journal of psychology.

[31]  P E Roland,et al.  Cross-Modal Transfer of Information between the Tactile and the Visual Representations in the Human Brain: A Positron Emission Tomographic Study , 1998, The Journal of Neuroscience.

[32]  Amir Amedi,et al.  A Putative Model of Multisensory Object Representation , 2009, Brain Topography.

[33]  J C Mazziotta,et al.  Automated labeling of the human brain: A preliminary report on the development and evaluation of a forward‐transform method , 1997, Human brain mapping.

[34]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[35]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[36]  Rainer Goebel,et al.  fMR-adaptation indicates selectivity to audiovisual content congruency in distributed clusters in human superior temporal cortex , 2010, BMC Neuroscience.

[37]  Thomas W James,et al.  Enhanced effectiveness in visuo‐haptic object‐selective brain regions with increasing stimulus salience , 2009, Human brain mapping.

[38]  Susan J. Lederman,et al.  Extracting object properties through haptic exploration. , 1993, Acta psychologica.

[39]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[40]  Paul J. Laurienti,et al.  Semantic congruence is a critical factor in multisensory behavioral performance , 2004, Experimental Brain Research.