Dorsal and Ventral Cortical Pathways for Visuo-haptic Shape Integration Revealed Using fMRI

Two sensory streams theories have had an important influence on sensory and sensorimotor research for the past several decades. Here we apply the perspective of two sensory streams to interactions between visual and haptic object shape processes. We specifically focus on the presence and pattern of multisensory integration or neuronal convergence in dorsal action pathways and ventral perception pathways. To investigate integration of visual and haptic processing streams, we assessed potential sites of visuo-haptic integration for a phenomenon called inverse effectiveness, that is, increased multisensory gain with decreasing stimulus salience. Unexpectedly, the opposite pattern, which we called enhanced effectiveness, was found. Nevertheless, finding enhanced effectiveness implies neuronal convergence of visual and haptic inputs in regions considered part of separable dorsal action and ventral perception visuo-haptic processing pathways.

[1]  A. Yamadori,et al.  The Influence of Misnaming on Object Recognition: A Case of Multimodal Agnosia , 2001, Cortex.

[2]  Doris Y. Tsao,et al.  Neuroimaging Weighs In: Humans Meet Macaques in “Primate” Visual Cortex , 2003, The Journal of Neuroscience.

[3]  Á. Pascual-Leone,et al.  The metamodal organization of the brain. , 2001, Progress in brain research.

[4]  B. Argall,et al.  Unraveling multisensory integration: patchy organization within human STS multisensory cortex , 2004, Nature Neuroscience.

[5]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[6]  H. Freund,et al.  Tactile apraxia: unimodal apractic disorder of tactile object exploration associated with parietal lobe lesions. , 2001, Brain : a journal of neurology.

[7]  Volkmar Glauche,et al.  Ventral and dorsal pathways for language , 2008, Proceedings of the National Academy of Sciences.

[8]  Michael S. Beauchamp,et al.  Statistical criteria in fMRI studies of multisensory integration , 2005, Neuroinformatics.

[9]  Michael S Beauchamp,et al.  See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex , 2005, Current Opinion in Neurobiology.

[10]  Scott T. Grafton,et al.  Feeling with the mind's eye , 1997, Neuroreport.

[11]  Geoffrey P. Bingham,et al.  Calibrating grasp size and reach distance: interactions reveal integral organization of reaching-to-grasp movements , 2008, Experimental Brain Research.

[12]  H. Freund,et al.  Sensorimotor disturbances in patients with lesions of the parietal cortex. , 1989, Brain : a journal of neurology.

[13]  P. Courtheoux,et al.  [Visual and tactile agnosia]. , 1984, Revue neurologique.

[14]  Craig E. L. Stark,et al.  When zero is not zero: The problem of ambiguous baseline conditions in fMRI , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[15]  K. Sathian,et al.  Multisensory cortical processing of object shape and its relation to mental imagery , 2004, Cognitive, affective & behavioral neuroscience.

[16]  T. R. Jordan,et al.  Perception and action in 'visual form agnosia'. , 1991, Brain : a journal of neurology.

[17]  B. Stein,et al.  Interactions among converging sensory inputs in the superior colliculus. , 1983, Science.

[18]  G. Fink,et al.  REVIEW: The functional organization of the intraparietal sulcus in humans and monkeys , 2005, Journal of anatomy.

[19]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[20]  D. Poeppel,et al.  Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language , 2004, Cognition.

[21]  Christopher A. Buneo,et al.  Direct visuomotor transformations for reaching , 2002, Nature.

[22]  Randall Stilla,et al.  Selective visuo‐haptic processing of shape and texture , 2008, Human brain mapping.

[23]  Keiji Tanaka,et al.  Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. , 1988, Journal of neurophysiology.

[24]  A. Georgopoulos,et al.  Parietal cortex neurons of the monkey related to the visual guidance of hand movement , 1990, Experimental Brain Research.

[25]  A. Milner.  Perception and action in visual form agnosia , 2000 .

[26]  Robert L. Goldstone,et al.  Perceptual Learning from Cross-modal Feedback , 1997 .

[27]  C Dohle,et al.  Human anterior intraparietal area subserves prehension , 1998, Neurology.

[28]  B. Stein,et al.  Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. , 1986, Journal of neurophysiology.

[29]  A. Amedi,et al.  Functional imaging of human crossmodal identification and object recognition , 2005, Experimental Brain Research.

[30]  T. James,et al.  The neural basis of haptic object processing. , 2007, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[31]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[32]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[33]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[34]  Thomas W James,et al.  Enhanced effectiveness in visuo‐haptic object‐selective brain regions with increasing stimulus salience , 2009, Human brain mapping.

[35]  Amir Amedi,et al.  Multisensory visual–tactile object related network in humans: insights gained using a novel crossmodal adaptation approach , 2009, Experimental Brain Research.

[36]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[37]  J. Maunsell,et al.  Extraretinal representations in area V4 in the macaque monkey , 1991, Visual Neuroscience.

[38]  Claude Alain,et al.  Assessing the auditory dual-pathway model in humans , 2004, NeuroImage.

[39]  R. D. Easton,et al.  Transfer between vision and haptics: Memory for 2-D patterns and 3-D objects , 1997 .

[40]  Paul J Laurienti,et al.  Age-related multisensory enhancement in a simple audiovisual detection task , 2007, Neuroreport.

[41]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[42]  M. Meredith,et al.  On the neuronal basis for multisensory convergence: a brief overview. , 2002, Brain research. Cognitive brain research.

[43]  K. Zilles,et al.  Crossmodal Processing of Object Features in Human Anterior Intraparietal Cortex An fMRI Study Implies Equivalencies between Humans and Monkeys , 2002, Neuron.

[44]  Ryan A. Stevenson,et al.  Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects , 2007, Experimental Brain Research.

[45]  Malcolm P. Young,et al.  Objective analysis of the topological organization of the primate cortical visual system , 1992, Nature.

[46]  Ravi S. Menon,et al.  The effects of visual object priming on brain activation before and after recognition , 2000, Current Biology.

[47]  N. Logothetis,et al.  Integration of Local Features into Global Shapes Monkey and Human fMRI Studies , 2003, Neuron.

[48]  T. Hendler,et al.  Convergence of visual and tactile shape processing in the human lateral occipital complex. , 2002, Cerebral cortex.

[49]  P. Roland,et al.  Shape and roughness activate different somatosensory areas in the human brain. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[50]  Xiaoping Hu,et al.  Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception , 2007, Neuropsychologia.

[51]  Soledad Ballesteros,et al.  Touch and Blindness: Psychology and Neuroscience , 2005 .

[52]  Scott T. Grafton,et al.  Involvement of visual cortex in tactile discrimination of orientation , 1999, Nature.

[53]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[54]  Roberta L. Klatzky,et al.  What vs. where in touch: an fMRI study , 2005, NeuroImage.

[55]  N. Kanwisher,et al.  Neuroimaging of cognitive functions in human parietal cortex , 2001, Current Opinion in Neurobiology.

[56]  H. C. Dijkerman,et al.  Somatosensory processes subserving perception and action , 2007, Behavioral and Brain Sciences.

[57]  Amir Amedi,et al.  A Putative Model of Multisensory Object Representation , 2009, Brain Topography.

[58]  P. Lennie.  Single Units and Visual Cortical Organization , 1998, Perception.

[59]  Scott T. Grafton,et al.  Cortical topography of human anterior intraparietal cortex active during visually guided grasping. , 2005, Brain research. Cognitive brain research.

[60]  H. Sakata,et al.  Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. , 2000, Journal of neurophysiology.

[61]  G K Humphrey,et al.  The Role of Surface Information in Object Recognition: Studies of a Visual Form Agnosic and Normal Subjects , 1994, Perception.

[62]  H. Bülthoff,et al.  Viewpoint Dependence in Visual and Haptic Object Recognition , 2001, Psychological science.

[63]  Paul J. Laurienti,et al.  On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies , 2005, Experimental Brain Research.

[64]  M. Goodale,et al.  The visual brain in action , 1995 .

[65]  Maurizio Corbetta,et al.  The McCollough effect reveals orientation discrimination in a case of cortical blindness , 1995, Current Biology.

[66]  R D Easton,et al.  Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[67]  Chris I. Baker,et al.  Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions , 2005, Journal of Cognitive Neuroscience.

[68]  M. Wallace,et al.  Enhanced multisensory integration in older adults , 2006, Neurobiology of Aging.

[69]  Kenneth F. Valyear,et al.  Human parietal cortex in action , 2006, Current Opinion in Neurobiology.

[70]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[71]  J. Culham,et al.  The role of parietal cortex in visuomotor control: What have we learned from neuroimaging? , 2006, Neuropsychologia.

[72]  L. Benevento,et al.  Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey , 1977, Experimental Neurology.

[73]  Hideko F. Norman,et al.  The visual and haptic perception of natural object shape , 2010 .

[74]  Leslie G. Ungerleider,et al.  Organization of visual inputs to the inferior temporal and posterior parietal cortex in macaques , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[75]  Susan J. Lederman,et al.  Multisensory Activation of the Intraparietal Area When Classifying Grating Orientation: A Functional Magnetic Resonance Imaging Study , 2006, The Journal of Neuroscience.

[76]  K. Zilles,et al.  Hierarchical Processing of Tactile Shape in the Human Brain , 2001, Neuron.

[77]  M V Jeffreys,et al.  Letter: John Locke. , 1975, British medical journal.

[78]  M. Goodale,et al.  Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. , 2003, Brain : a journal of neurology.

[79]  J. Bullier,et al.  Anatomical segregation of two cortical visual pathways in the macaque monkey , 1990, Visual Neuroscience.

[80]  N. Kanwisher,et al.  The lateral occipital complex and its role in object recognition , 2001, Vision Research.

[81]  Kenneth M. Heilman,et al.  Multimodal agnosia after unilateral left hemisphere lesion , 1986, Neurology.

[82]  L. Jakobson,et al.  A neurological dissociation between perceiving objects and grasping them , 1991, Nature.

[83]  Leslie G. Ungerleider.  Two cortical visual systems , 1982 .

[84]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[85]  Ryan A. Stevenson,et al.  Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition , 2009, NeuroImage.

[86]  S. Ballesteros,et al.  Implicit and Explicit Memory for Visual and Haptic Objects: Cross-Modal Priming Depends on Structural Descriptions , 1999 .

[87]  Shy Shoham,et al.  Neural substrates of tactile object recognition: An fMRI study , 2004, Human brain mapping.