What neuroimaging tells us about sensory substitution

A major question in the field of sensory substitution concerns the nature of the perception generated by sensory substitution prostheses. Is the perception determined by the nature of the substitutive modality or is it determined by the nature of the information transmitted by the device? Is it a totally new, amodal, perception? This paper reviews the recent neuroimaging studies which have investigated the neural bases of sensory substitution. The detailed analysis of available results led us to propose a general scheme of the neural mechanisms underlying sensory substitution. Two different main processes may be responsible for the visual area recruitment observed in the different studies: cross-modality and mental (visual) imagery. Based on our results analysis, we propose that cross-modality is the predominant process in early blind subjects whereas mental imagery is predominant in blindfolded sighted subjects. This model implies that, with training, sensory substitution mainly induces visual-like perception in sighted subjects and mainly auditory or tactile perception in blind subjects. This framework leads us to make some predictions that could easily be tested.

[1]  M. Peterson,et al.  Mental images can be ambiguous: Reconstruals and reference-frame reversals , 1992, Memory & cognition.

[2]  J. Cronly-Dillon,et al.  The perception of visual images encoded in musical form: a study in cross-modality information transfer , 1999, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[3]  M. Kiyosawa,et al.  Auditory Triggered Mental Imagery of Shape Involves Visual Association Areas in Early Blind Humans , 2001, NeuroImage.

[4]  Kenith V. Sobel,et al.  PSYCHOLOGICAL SCIENCE Research Article Neural Synergy Between Kinetic Vision and Touch , 2022 .

[5]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[6]  Sandra M. Sanabria-Bohórquez,et al.  Occipital Activation by Pattern Recognition in the Early Blind Using Auditory Substitution for Vision , 2001, NeuroImage.

[7]  Laurent Renier,et al.  Vertical-horizontal illusion present for sighted but not early blind humans using auditory substitution of vision , 2006, Perception & psychophysics.

[8]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[9]  M. Wanet-Defalque,et al.  Auditory coding of visual patterns for the blind. , 1999, Perception.

[10]  Annick Vanlierde,et al.  Occipito-parietal cortex activation during visuo-spatial imagery in early blind humans , 2003, NeuroImage.

[11]  Colline Poirier,et al.  Auditory motion perception activates visual motion areas in early blind subjects , 2006, NeuroImage.

[12]  P. Schyns,et al.  Usage of spatial scales for the categorization of faces, objects, and scenes , 2001, Psychonomic bulletin & review.

[13]  G. S. Marmor,et al.  Mental rotation by the blind: does mental rotation depend on visual imagery? , 1976, Journal of experimental psychology. Human perception and performance.

[14]  Y. Mauss,et al.  Blindness and brain plasticity: contribution of mental imagery? An fMRI study. , 2004, Brain research. Cognitive brain research.

[15]  Alvaro Pascual-Leone,et al.  Behavioral and neuroplastic changes in the blind: evidence for functionally relevant cross-modal interactions , 2004, Journal of Physiology-Paris.

[16]  Albert Gjedde,et al.  Erratum: Transcranial magnetic stimulation of the visual cortex induces somatotopically organized qualia in blind subjects (Proceedings of the National Academy of Sciences of the United States of America (August 29, 2006) 103, 35 (13256-13260) DOI: 10.1073/pnas.0602925103) , 2006 .

[17]  N H Kerr,et al.  The role of vision in "visual imagery" experiments: evidence from the congenitally blind. , 1983, Journal of experimental psychology. General.

[18]  M. Denis,et al.  Functional Anatomy of Spatial Mental Imagery Generated from Verbal Instructions , 1996, The Journal of Neuroscience.

[19]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[20]  Anne Bol,et al.  Cross-modal activation of visual cortex during depth perception using auditory substitution of vision , 2005, NeuroImage.

[21]  B Hughes,et al.  Perceptual learning of spatiotemporal events: evidence from an unfamiliar modality. , 1989, Journal of experimental psychology. Human perception and performance.

[22]  C. Trullemans,et al.  A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution , 1998, IEEE Transactions on Biomedical Engineering.

[23]  S. Kosslyn.  Measuring the visual angle of the mind's eye , 1978, Cognitive Psychology.

[24]  W.J. Tompkins,et al.  Electrotactile and vibrotactile displays for sensory substitution systems , 1991, IEEE Transactions on Biomedical Engineering.

[25]  A. Volder,et al.  Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects , 2007, Neuropsychologia.

[26]  Leslie G. Ungerleider,et al.  ‘What’ and ‘where’ in the human brain , 1994, Current Opinion in Neurobiology.

[27]  R Bruyer,et al.  The Ponzo Illusion with Auditory Substitution of Vision in Sighted and Early-Blind Subjects , 2005, Perception.

[28]  R. Malach,et al.  Early ‘visual’ cortex activation correlates with superior verbal memory performance in the blind , 2003, Nature Neuroscience.

[29]  C. Veraart,et al.  Assessment of sensory substitution prosthesis potentialities in minimalist conditions of learning , 2006 .

[30]  J. Haxby,et al.  The effect of visual experience on the development of functional architecture in hMT+. , 2007, Cerebral cortex.

[31]  C. Veraart,et al.  Functional Cerebral Reorganization for Auditory Spatial Processing and Auditory Substitution of Vision in Early Blind Subjects , 2006 .

[32]  M. Ptito,et al.  Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. , 2005, Brain : a journal of neurology.

[33]  S. Kosslyn,et al.  When is early visual cortex activated during visual mental imagery? , 2003, Psychological bulletin.

[34]  S. Kosslyn,et al.  Topographical representations of mental images in primary visual cortex , 1995, Nature.

[35]  Marie-Chantal Wanet-Defalque,et al.  Auditory substitution of vision: pattern recognition by the blind , 2001 .

[36]  D. Navon,et al.  Global precedence and response activation: Evidence from LRPs , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[37]  R. Zatorre,et al.  A Functional Neuroimaging Study of Sound Localization: Visual Cortex Activity Predicts Performance in Early-Blind Individuals , 2005, PLoS biology.

[38]  Paul Bach-y-Rita,et al.  Brain plasticity: ‘visual’ acuity of blind persons via the tongue , 2001, Brain Research.

[39]  A. Vanlierde,et al.  Specific activation of the V5 brain area by auditory motion processing: an fMRI study. , 2005, Brain research. Cognitive brain research.