Cross-modal activation of visual cortex during depth perception using auditory substitution of vision

Previous neuroimaging studies identified multimodal brain areas in the visual cortex that are specialized for processing specific information, such as visual-haptic object recognition. Here, we test whether visual brain areas are involved in depth perception when auditory substitution of vision is used. Nine sighted volunteers were trained blindfolded to use a prosthesis substituting vision with audition both to recognize two-dimensional figures and to estimate distance of an object in a real three-dimensional environment. Using positron emission tomography, regional cerebral blood flow was assessed while the prosthesis was used to explore virtual 3D images; subjects focused either on 2D features (target search) or on depth (target distance comparison). Activation foci were found in visual association areas during both the target search task, which recruited the occipito-parietal cortex, and the depth perception task, which recruited occipito-parietal and occipito-temporal areas. This indicates that some brain areas of the visual cortex are relatively multimodal and may be recruited for depth processing via a sense other than vision.

[1]  A. Cowey,et al.  The role of the parietal cortex in visual attention—hemispheric asymmetries and the effects of learning: a magnetic stimulation study , 1998, Neuropsychologia.

[2]  M. Heller,et al.  Production and Interpretation of Perspective Drawings by Blind and Sighted People , 1996, Perception.

[3]  Mark W Greenlee,et al.  BOLD response in dorsal areas varies with relative disparity level , 2004, Neuroreport.

[4]  Leslie G. Ungerleider,et al.  Dissociation of object and spatial visual processing pathways in human extrastriate cortex. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[5]  J. Sergent,et al.  Functional neuroanatomy of face and object processing. A positron emission tomography study. , 1992, Brain : a journal of neurology.

[6]  Scott T. Grafton,et al.  Automated image registration: I. General methods and intrasubject, intramodality validation. , 1998, Journal of computer assisted tomography.

[7]  P. Bach-y-Rita Brain mechanisms in sensory substitution , 1972 .

[8]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[9]  H. Bülthoff,et al.  Representation of the perceived 3-D object shape in the human lateral occipital complex. , 2003, Cerebral cortex.

[10]  C. Trevarthen,et al.  Two mechanisms of vision in primates , 1968, Psychologische Forschung.

[11]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[12]  Jacob Kline,et al.  Handbook of biomedical engineering , 1987 .

[13]  Alan Cowey,et al.  Temporal aspects of visual search studied by transcranial magnetic stimulation , 1997, Neuropsychologia.

[14]  M. Mintun,et al.  Enhanced Detection of Focal Brain Responses Using Intersubject Averaging and Change-Distribution Analysis of Subtracted PET Images , 1988, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[15]  Leslie G. Ungerleider Two cortical visual systems , 1982 .

[16]  G. Orban,et al.  Selectivity for 3D shape that reveals distinct areas within macaque inferior temporal cortex. , 2000, Science.

[17]  R Vogels,et al.  Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[18]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[19]  John R. Votaw,et al.  Task-specific recruitment of dorsal and ventral visual areas during tactile perception , 2004, Neuropsychologia.

[20]  K Wienhard,et al.  The ECAT EXACT HR: Performance of a New High Resolution Positron Scanner , 1994, Journal of computer assisted tomography.

[21]  David J. Fleet,et al.  Human cortical activity correlates with stereoscopic depth perception. , 2001, Journal of neurophysiology.

[22]  Scott T. Grafton,et al.  Feeling with the mind's eye , 1997, Neuroreport.

[23]  Arno Villringer,et al.  Parietal activation during visual search in the absence of multiple distractors , 2003, Neuroreport.

[24]  Christian N L Olivers,et al.  Spatiotemporal segregation in visual search: evidence from parietal lesions. , 2004, Journal of experimental psychology. Human perception and performance.

[25]  M. Taira,et al.  Cortical Areas Related to Attention to 3D Surface Structures Based on Shading: An fMRI Study , 2001, NeuroImage.

[26]  D. V. van Essen,et al.  Processing of color, form and disparity information in visual areas VP and V2 of ventral extrastriate cortex in the macaque monkey , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  Cédric Laloyaux,et al.  Perception of visual illusions with a sensory substitution system , 2003 .

[28]  M. Wanet-Defalque,et al.  Auditory coding of visual patterns for the blind. , 1999, Perception.

[29]  Makoto Kato,et al.  Processing of shape defined by disparity in monkey inferior temporal cortex. , 2001 .

[30]  G. Schneider Two visual systems. , 1969, Science.

[31]  Bruce Cumming Stereopsis: Where Depth is Seen , 2002, Current Biology.

[32]  Sandra M. Sanabria-Bohórquez,et al.  Occipital Activation by Pattern Recognition in the Early Blind Using Auditory Substitution for Vision , 2001, NeuroImage.

[33]  B. Gulyás,et al.  Binocular disparity discrimination in human cerebral cortex: functional anatomy by positron emission tomography. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Toshio Inui,et al.  Neural substrates for depth perception of the Necker cube; a functional magnetic resonance imaging study in human subjects , 2000, Neuroscience Letters.

[35]  G. DeAngelis,et al.  Organization of Disparity-Selective Neurons in Macaque Area MT , 1999, The Journal of Neuroscience.

[36]  J. G. Wallace,et al.  Recovery from early blindness : a case study , 1963 .

[37]  M. Corbetta,et al.  Superior Parietal Cortex Activation During Spatial Attention Shifts and Visual Feature Conjunction , 1995, Science.

[38]  R T Knight,et al.  Cortical substrates supporting visual search in humans. , 1991, Cerebral cortex.

[39]  T. Hendler,et al.  Convergence of visual and tactile shape processing in the human lateral occipital complex. , 2002, Cerebral cortex.

[40]  A. Treisman,et al.  Parietal contributions to visual feature binding: evidence from a patient with bilateral lesions , 1995, Science.

[41]  R. Held Dissociation of visual functions by deprivation and rearrangement , 1968 .

[42]  M. Wallace,et al.  A revised view of sensory cortical parcellation , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Alan Cowey,et al.  Does parietal cortex contribute to feature binding? , 1999, Neuropsychologia.

[44]  Alan C. Evans,et al.  Localization and lateralization of stereoscopic processing in the human brain. , 1993, Neuroreport.

[45]  Vincent Walsh,et al.  Distinct neural substrates for visual search amongst spatial versus temporal distractors. , 2003, Brain research. Cognitive brain research.

[46]  A. Villringer,et al.  Involvement of the human frontal eye field and multiple parietal areas in covert visual selection during conjunction search , 2000, The European journal of neuroscience.

[47]  I. Fujita,et al.  Disparity selectivity of neurons in monkey inferior temporal cortex. , 2000, Journal of neurophysiology.

[48]  Norimichi Kitagawa,et al.  Hearing visual motion in depth , 2002, Nature.

[50]  K. Sathian,et al.  Multisensory cortical processing of object shape and its relation to mental imagery , 2004, Cognitive, affective & behavioral neuroscience.

[51]  Gian F. Poggio Mechanisms of Stereopsis in Monkey Visual Cortex , 1995 .

[52]  R. Mansfield,et al.  Analysis of visual behavior , 1982 .

[53]  B. Macq,et al.  Interactive delineation of brain sulci and their merging into functional PET images , 1995, 1995 IEEE Nuclear Science Symposium and Medical Imaging Conference Record.

[54]  D. L. Adams,et al.  Functional organization of macaque V3 for stereoscopic depth. , 2001, Journal of neurophysiology.

[55]  David L. Sheinberg,et al.  Visual object recognition. , 1996, Annual review of neuroscience.

[56]  J C Mazziotta,et al.  Automated image registration: II. Intersubject validation of linear and nonlinear models. , 1998, Journal of computer assisted tomography.

[57]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[58]  J. Mazziotta,et al.  A Noninvasive Positron Computed Tomography Technique Using Oxygen-15-Labeled Water for the Evaluation of Neurobehavioral Task Batteries , 1985, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[59]  Karl J. Friston,et al.  A multimodal language region in the ventral visual pathway , 1998, Nature.

[60]  Keiji Tanaka,et al.  Coding visual images of objects in the inferotemporal cortex of the macaque monkey. , 1991, Journal of neurophysiology.

[61]  R. Desimone,et al.  Shape recognition and inferior temporal neurons. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[62]  C. Trullemans,et al.  A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution , 1998, IEEE Transactions on Biomedical Engineering.

[63]  J. Rauschecker Compensatory plasticity and sensory substitution in the cerebral cortex , 1995, Trends in Neurosciences.

[64]  C E Connor,et al.  Disparity tuning in macaque area V4 , 2001, Neuroreport.

[65]  Doris Y. Tsao,et al.  Response to Tyler: Representation of stereoscopic structure in human and monkey cortex , 2004, Trends in Neurosciences.

[66]  C Veraart,et al.  Neurophysiological approach to the design of visual prostheses: a theoretical discussion. , 1989, Journal of medical engineering & technology.

[67]  Scott T. Grafton,et al.  Involvement of visual cortex in tactile discrimination of orientation , 1999, Nature.

[68]  Gregory J. Zelinsky,et al.  Effect of parietal lobe lesions on saccade targeting and spatial memory in a naturalistic visual search task , 2003, Neuropsychologia.

[69]  Doris Y. Tsao,et al.  Stereopsis Activates V3A and Caudal Intraparietal Areas in Macaques and Humans , 2003, Neuron.

[70]  C. Connor,et al.  Shape representation in area V4: position-specific tuning for boundary conformation. , 2001, Journal of neurophysiology.

[71]  Nikos K. Logothetis,et al.  Three-Dimensional Shape Representation in Monkey Cortex , 2002, Neuron.

[72]  John C Gore,et al.  The role of the parietal cortex in visual feature binding , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[73]  John H. R. Maunsell,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. , 1983, Journal of neurophysiology.

[74]  E. Deibert,et al.  Neural pathways in tactile object recognition , 1999, Neurology.

[75]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[76]  M. Kiyosawa,et al.  Auditory Triggered Mental Imagery of Shape Involves Visual Association Areas in Early Blind Humans , 2001, NeuroImage.

[77]  K. Kaczmarek Sensory Augmentation and Substitution , 1999 .

[78]  Rüdiger J. Seitz,et al.  A fronto-parietal circuit for tactile object discrimination: an event-related fMRI study , 2003, NeuroImage.

[79]  R Perez,et al.  Neural mechanisms underlying stereoscopic vision , 1998, Progress in Neurobiology.

[80]  Tammo Houtgast,et al.  Auditory distance perception in rooms , 1999, Nature.

[81]  G. Orban,et al.  Three-Dimensional Shape Coding in Inferior Temporal Cortex , 2000, Neuron.