Shape Discrimination Using the Tongue: Implications for a Visual-to-Tactile Sensory Substitution Device

Sensory substitution devices have the potential to provide individuals with visual impairments with more information about their environments, which may help them recognize objects and achieve more independence in their daily lives. However, many of these devices may require extensive training and might be limited in the amount of information that they can convey. We tested the effectiveness and assessed some of the limitations of the BrainPort device, which provides stimulation through a 20 × 20 electrode grid array on the tongue. Across five experiments, including one with blind individuals, we found that subjects were unable to accurately discriminate between simple shapes as well as different line orientations that were briefly presented on the tongue, even after 300 trials of practice with the device. These experiments indicate that such a minimal training regimen with the BrainPort is not sufficient for object recognition, raising serious concerns about the usability of this device without extensive training.

[1]  R. Malach,et al.  Negative BOLD Differentiates Visual Imagery and Perception , 2005, Neuron.

[2]  K. O. Johnson,et al.  Human tactile pattern recognition: active versus passive touch, velocity effects, and patterns of confusion. , 1991, Journal of neurophysiology.

[3]  R. Zatorre,et al.  A positron emission tomography study during auditory localization by late-onset blind individuals , 2006, Neuroreport.

[4]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[5]  M. Raichle,et al.  Adaptive changes in early and late blind: a fMRI study of Braille reading. , 2002, Journal of neurophysiology.

[6]  C. Y. Wan,et al.  Congenital blindness leads to enhanced vibrotactile perception , 2010, Neuropsychologia.

[7]  Mary S. Binseel,et al.  The Tactile Modality: A Review of Tactile Sensitivity and Human Tactile Interfaces , 2007 .

[8]  Shinsuke Shimojo,et al.  Auditory Sensory Substitution is Intuitive and Automatic with Texture Stimuli , 2015, Scientific Reports.

[9]  Paul Bach-y-Rita,et al.  Brain plasticity: ‘visual’ acuity of blind persons via the tongue , 2001, Brain Research.

[10]  Kristoffer Hougaard Madsen,et al.  Neural correlates of virtual route recognition in congenital blindness , 2010, Proceedings of the National Academy of Sciences.

[11]  M. Péladeau-Pigeon,et al.  Age and strength influences on lingual tactile acuity. , 2014, Journal of texture studies.

[12]  G. Vandewalle,et al.  Impact of blindness onset on the functional organization and the connectivity of the occipital cortex. , 2013, Brain : a journal of neurology.

[13]  Á. Pascual-Leone,et al.  Braille character discrimination in blindfolded human subjects , 2002, Neuroreport.

[14]  Laurence R. Harris,et al.  Plasticity in Sensory Systems , 2012 .

[15]  Zhigang Zhu,et al.  Evaluating Crowd Sourced Navigation for the Visually Impaired in a Virtual Environment , 2015, 2015 IEEE International Conference on Mobile Services.

[16]  Charles Spence,et al.  The skin as a medium for sensory substitution. , 2014, Multisensory research.

[17]  M. Ptito,et al.  Beyond visual, aural and haptic movement perception: hMT+ is activated by electrotactile motion stimulation of the tongue in sighted and in congenitally blind individuals , 2010, Brain Research Bulletin.

[18]  M. Ptito,et al.  Recruitment of the middle temporal area by tactile motion in congenital blindness , 2009, Neuroreport.

[19]  A. Volder,et al.  Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects , 2007, Neuropsychologia.

[20]  David C. Reutens,et al.  Early but not late-blindness leads to enhanced auditory perception , 2010, Neuropsychologia.

[21]  Mandy Kendrick,et al.  Tasting the light. , 2009, Scientific American.

[22]  M. Hallett,et al.  Period of susceptibility for cross‐modal plasticity in the blind , 1999, Annals of neurology.

[23]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[24]  H. Stam,et al.  The Elicitation of Visual Hallucinations via Brief Instructions in a Normal Sample , 1979, The Journal of nervous and mental disease.

[25]  Shachar Maidenbaum,et al.  EyeMusic: Introducing a "visual" colorful experience for the blind using auditory sensory substitution. , 2014, Restorative neurology and neuroscience.

[26]  Zhigang Zhu,et al.  Designing and testing wearable range‐vibrotactile devices , 2013 .

[27]  Andrew J. R. Simpson,et al.  Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli? , 2015, Front. Psychol..

[28]  K O Johnson,et al.  The limit of tactile spatial resolution in humans , 1994, Neurology.

[29]  Zhigang Zhu,et al.  Designing a Virtual Environment to Evaluate Multimodal Sensors for Assisting the Visually Impaired , 2012, ICCHP.

[30]  Charles Laymon,et al.  Successful tactile based visual sensory substitution use functions independently of visual pathway integrity , 2014, Front. Hum. Neurosci..

[31]  Manabu Honda,et al.  Critical Period for Cross-Modal Plasticity in Blind Humans: A Functional MRI Study , 2002, NeuroImage.

[32]  Amy C Nau,et al.  Acquisition of Visual Perception in Blind Adults Using the BrainPort Artificial Vision Device. , 2014, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[33]  Laurence R. Harris,et al.  Plasticity in Sensory Systems: List of Contributors , 2012 .

[34]  Amir Amedi,et al.  Increased functional connectivity between language and visually deprived areas in late and partial blindness , 2016, NeuroImage.

[35]  M. Ptito,et al.  Crossmodal Recruitment of the Ventral Visual Stream in Congenital Blindness , 2012, Neural plasticity.

[36]  Ron Kupers,et al.  Tactile–‘visual’ acuity of the tongue in early blind individuals , 2007, Neuroreport.

[37]  G. Essick,et al.  Low-threshold mechanoreceptive afferents in the human lingual nerve. , 1997, Journal of neurophysiology.

[38]  E. Deibert,et al.  Neural pathways in tactile object recognition , 1999, Neurology.

[39]  M. Raichle,et al.  Adaptive changes in early and late blind: a FMRI study of verb generation to heard nouns. , 2002, Journal of neurophysiology.

[40]  Shachar Maidenbaum,et al.  Author's Personal Copy Neuroscience and Biobehavioral Reviews Sensory Substitution: Closing the Gap between Basic Research and Widespread Practical Visual Rehabilitation Author's Personal Copy , 2022 .

[41]  M. Ptito,et al.  Cross-modal plasticity revealed by electrotactile stimulation of the tongue in the congenitally blind. , 2005, Brain : a journal of neurology.

[42]  Amir Amedi,et al.  Negative BOLD in Sensory Cortices During Verbal Memory: A Component in Generating Internal Representations? , 2009, Brain Topography.

[43]  Christopher Fisher,et al.  Clinical Tests of Ultra-Low Vision Used to Evaluate Rudimentary Visual Perceptions Enabled by the BrainPort Vision Device. , 2013, Translational vision science & technology.

[44]  G. Essick,et al.  A letter-recognition task to assess lingual tactile acuity. , 1999, Journal of oral and maxillofacial surgery : official journal of the American Association of Oral and Maxillofacial Surgeons.

[45]  Ron Kupers,et al.  Navigation with a sensory substitution device in congenitally blind individuals , 2011, Neuroreport.

[46]  M. Hallett,et al.  Activation of the primary visual cortex by Braille reading in blind subjects , 1996, Nature.

[47]  G. Lamb.  Tactile discrimination of textured surfaces: psychophysical performance measurements in humans. , 1983, The Journal of physiology.

[48]  Marco Santello,et al.  Electrotactile stimulation on the tongue: Intensity perception, discrimination, and cross-modality estimation , 2009, Somatosensory & motor research.

[49]  Shachar Maidenbaum,et al.  Perception of Graphical Virtual Environments by Blind Users via Sensory Substitution , 2016, PloS one.

[50]  Scott T. Grafton,et al.  Feeling with the mind's eye , 1997, Neuroreport.

[51]  Peter B. L. Meijer,et al.  Visual experiences in the blind induced by an auditory sensory substitution device , 2010, Consciousness and Cognition.

[52]  R. Klatzky,et al.  There's more to touch than meets the eye: The salience of object attributes for haptics with and without vision. , 1987 .

[53]  P. Bach-y-Rita,et al.  Form perception with a 49-point electrotactile stimulus array on the tongue: a technical note. , 1998, Journal of rehabilitation research and development.

[54]  J. Gibson.  Observations on active touch. , 1962, Psychological review.