Can you hear shapes you touch?

Shape is an inherent property of objects existing in both vision and touch but not audition. Can shape then be represented by sound artificially? It has previously been shown that sound can convey visual information by means of image-to-sound coding, but whether sound can code tactile information is not clear. Blindfolded sighted individuals were trained to recognize tactile spatial information using sounds mapped from abstract shapes. After training, subjects were able to match auditory input to tactually discerned shapes and showed generalization to novel auditory–tactile pairings. Furthermore, they showed complete transfer to novel visual shapes, despite the fact that training did not involve any visual exposure. In addition, we found enhanced tactile acuity specific to the training stimuli. The present study demonstrates that as long as tactile space is coded in a systematic way, shape can be conveyed via a medium that is not spatial, suggesting a metamodal representation.

[1]  Jung-Kyong Kim,et al.  Generalized Learning of Visual-to-auditory Substitution in Sighted Individuals , 2008 .

[2]  C. Carello,et al.  Perception of Object Length by Sound , 1998 .

[3]  Anne Bol,et al.  Cross-modal activation of visual cortex during depth perception using auditory substitution of vision , 2005, NeuroImage.

[4]  C. Gilbert,et al.  Perceptual learning of spatial localization: specificity for orientation, position, and context. , 1997, Journal of neurophysiology.

[5]  Hideko F. Norman,et al.  The visual and haptic perception of natural object shape , 2010 .

[6]  E. Deibert,et al.  Neural pathways in tactile object recognition , 1999, Neurology.

[7]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[8]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[9]  G Meinhardt.  Learning to discriminate simple sinusoidal gratings is task specific , 2002, Psychological research.

[10]  M. Wanet-Defalque,et al.  Auditory coding of visual patterns for the blind. , 1999, Perception.

[11]  N. Qian,et al.  Learning and adaptation in a recurrent model of V1 orientation selectivity. , 2003, Journal of neurophysiology.

[12]  K O Johnson,et al.  The limit of tactile spatial resolution in humans , 1994, Neurology.

[13]  M. Merzenich,et al.  Plasticity in the frequency representation of primary auditory cortex following discrimination training in adult owl monkeys , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  A. Fiorentini,et al.  Perceptual learning specific for orientation and spatial frequency , 1980, Nature.

[15]  M. Grassi.  Do we hear size or sound? Balls dropped on plates , 2005, Perception & psychophysics.

[16]  Scott T. Grafton,et al.  Involvement of visual cortex in tactile discrimination of orientation , 1999, Nature.

[17]  P. Bach-y-Rita.  Brain mechanisms in sensory substitution , 1972 .

[18]  R. Sekuler,et al.  Direction-specific improvement in motion discrimination , 1987, Vision Research.

[19]  S. Hochstein,et al.  Attentional control of early perceptual learning. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[20]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[21]  G. Recanzone,et al.  Topographic reorganization of the hand representation in cortical area 3b owl monkeys trained in a frequency-discrimination task. , 1992, Journal of neurophysiology.

[22]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[23]  Hideko F. Norman,et al.  Learning to Perceive Differences in Solid Shape through Vision and Touch , 2008, Perception.

[24]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[25]  A. Amedi,et al.  Functional imaging of human crossmodal identification and object recognition , 2005, Experimental Brain Research.

[26]  C. Gilbert,et al.  Perceptual learning and top-down influences in primary visual cortex , 2004, Nature Neuroscience.

[27]  Michael M Merzenich,et al.  Perceptual Learning Directs Auditory Cortical Map Reorganization through Top-Down Influences , 2006, The Journal of Neuroscience.

[28]  Á. Pascual-Leone,et al.  The metamodal organization of the brain. , 2001, Progress in brain research.

[29]  C. Trullemans,et al.  A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution , 1998, IEEE Transactions on Biomedical Engineering.

[30]  M. Mesulam,et al.  From sensation to cognition. , 1998, Brain : a journal of neurology.

[31]  J. O'Regan,et al.  Learning to Perceive with a Visuo — Auditory Substitution System: Localisation and Object Recognition with ‘The Voice’ , 2007, Perception.

[32]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[33]  Russell L. Martin,et al.  Specificity of perceptual learning in a frequency discrimination task. , 2000, The Journal of the Acoustical Society of America.

[34]  S. Ballesteros,et al.  Implicit and Explicit Memory for Visual and Haptic Objects: Cross-Modal Priming Depends on Structural Descriptions , 1999 .

[35]  C. Gilbert,et al.  Learning to see: experience and attention in primary visual cortex , 2001, Nature Neuroscience.

[36]  R Bruyer,et al.  The Ponzo Illusion with Auditory Substitution of Vision in Sighted and Early-Blind Subjects , 2005, Perception.

[37]  G. Calvert.  Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[38]  Scott T. Grafton,et al.  Feeling with the mind's eye , 1997, Neuroreport.