Multisensory cortical processing of object shape and its relation to mental imagery

Here, we used functional magnetic resonance imaging to investigate the multisensory processing of object shape in the human cerebral cortex and explored the role of mental imagery in such processing. Regions active bilaterally during both visual and haptic shape perception, relative to texture perception in the respective modality, included parts of the superior parietal gyrus, the anterior intraparietal sulcus, and the lateral occipital complex. Of these bimodal regions, the lateral occipital complexes preferred visual over haptic stimuli, whereas the parietal areas preferred haptic over visual stimuli. Whereas most subjects reported little haptic imagery during visual shape perception, experiences of visual imagery during haptic shape perception were common. Across subjects, ratings of the vividness of visual imagery strongly predicted the amount of haptic shape-selective activity in the right, but not in the left, lateral occipital complex. Thus, visual imagery appears to contribute to activation of some, but not all, visual cortical areas during haptic perception.

[1]  I ROCK,et al.  Vision and Touch: An Experimentally Created Conflict between the Two Senses , 1964, Science.

[2]  Phil Brodatz,et al.  Textures: A Photographic Album for Artists and Designers , 1966 .

[3]  D. F. Marks,et al.  Visual imagery differences in the recall of pictures. , 1973, British journal of psychology.

[4]  D. Freides,et al.  Human information processing and sensory modality: cross-modal functions, information complexity, memory, and deficit. , 1974, Psychological bulletin.

[5]  R. Nebes,et al.  Reliability and validity of some handedness questionnaire items. , 1974, Neuropsychologia.

[6]  Kenneth M. Heilman,et al.  Multimodal agnosia after unilateral left hemisphere lesion , 1986, Neurology.

[7]  R. Klatzky,et al.  There's more to touch than meets the eye: The salience of object attributes for haptics with and without vision. , 1987 .

[8]  R. Klatzky,et al.  Visual mediation and the haptic recognition of two-dimensional pictures of common objects , 1990, Perception & psychophysics.

[9]  J. Mugler,et al.  Three‐dimensional magnetization‐prepared rapid gradient‐echo imaging (3D MP RAGE) , 1990, Magnetic resonance in medicine.

[10]  J. Sergent,et al.  Functional neuroanatomy of face and object processing. A positron emission tomography study. , 1992, Brain : a journal of neurology.

[11]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Mark S. Cohen,et al.  Changes in cortical activity during mental rotation. A mapping study using functional MRI. , 1996, Brain : a journal of neurology.

[13]  Kavitha Srinivas,et al.  Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. , 1997 .

[14]  Scott T. Grafton,et al.  Feeling with the mind's eye , 1997, Neuroreport.

[15]  M. Petrides,et al.  Functional activation of the human brain during mental rotation , 1997, Neuropsychologia.

[16]  R. D. Easton,et al.  Transfer between vision and haptics: Memory for 2-D patterns and 3-D objects , 1997 .

[17]  S. Edelman,et al.  Human Brain Mapping 6:316–328(1998) � A Sequence of Object-Processing Stages Revealed by fMRI in the Human Occipital Lobe , 2022 .

[18]  Scott T. Grafton,et al.  Involvement of visual cortex in tactile discrimination of orientation , 1999, Nature.

[19]  R. J. Seitz,et al.  A fronto‐parietal circuit for object manipulation in man: evidence from an fMRI‐study , 1999, The European journal of neuroscience.

[20]  S. Ballesteros,et al.  Implicit and Explicit Memory for Visual and Haptic Objects: Cross-Modal Priming Depends on Structural Descriptions , 1999 .

[21]  K. Sathian,et al.  Tactile perception in developmental dyslexia: a psychophysical study using gratings , 1999, Neuropsychologia.

[22]  G. Orban,et al.  Attention Mechanisms in Visual SearchAn fMRI Study , 2000, Journal of Cognitive Neuroscience.

[23]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[24]  D. Gitelman,et al.  Functional Specificity of Superior Parietal Mediation of Spatial Shifting , 2001, NeuroImage.

[25]  K Sathian,et al.  Feeling with the mind's eye: the role of visual imagery in tactile perception. , 2001, Optometry and vision science : official publication of the American Academy of Optometry.

[26]  M. Kiyosawa,et al.  Auditory Triggered Mental Imagery of Shape Involves Visual Association Areas in Early Blind Humans , 2001, NeuroImage.

[27]  M. Preul The Human Brain: Surface, Blood Supply, and Three-Dimensional Sectional Anatomy , 2001 .

[28]  K. Zilles,et al.  Polymodal Motion Processing in Posterior Parietal and Premotor Cortex A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys , 2001, Neuron.

[29]  K. Zilles,et al.  Hierarchical Processing of Tactile Shape in the Human Brain , 2001, Neuron.

[30]  T. Hendler,et al.  Convergence of visual and tactile shape processing in the human lateral occipital complex. , 2002, Cerebral cortex.

[31]  G. Essick,et al.  Tactile motion activates the human middle temporal/V5 (MT/V5) complex , 2002, The European journal of neuroscience.

[32]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[33]  K. Zilles,et al.  Crossmodal Processing of Object Features in Human Anterior Intraparietal Cortex An fMRI Study Implies Equivalencies between Humans and Monkeys , 2002, Neuron.

[34]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[35]  C. Frith,et al.  Directing attention to locations and to sensory modalities: multiple levels of selective processing revealed with PET. , 2002, Cerebral cortex.

[36]  Kathleen S Rockland,et al.  Multisensory convergence in calcarine visual areas in macaque monkey. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[37]  Rüdiger J. Seitz,et al.  A fronto-parietal circuit for tactile object discrimination: an event-related fMRI study , 2003, NeuroImage.

[38]  C. Spence,et al.  Multisensory representation of limb position in human premotor cortex , 2003, Nature Neuroscience.

[39]  Minming Zhang,et al.  Neural networks active during tactile form perception: common and differential activity during macrospatial and microspatial tasks. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[40]  T. Hackett,et al.  Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[41]  John R. Votaw,et al.  Task-specific recruitment of dorsal and ventral visual areas during tactile perception , 2004, Neuropsychologia.

[42]  Kenith V. Sobel,et al.  PSYCHOLOGICAL SCIENCE Research Article Neural Synergy Between Kinetic Vision and Touch , 2022 .

[43]  S. Lederman,et al.  Haptic face recognition and prosopagnosia , 2004, Neuropsychologia.