Vision through other senses: Practical use of Sensory Substitution devices as assistive technology for visual rehabilitation

Visual-to-auditory Sensory Substitution Devices (SSDs) are non-invasive sensory aids that provide visual information to the blind via their functioning senses, such as audition. For years SSDs have been confined to laboratory settings, but we believe the time has come to use them also for their original purpose of real-world practical visual rehabilitation. Here we demonstrate this potential by presenting for the first time new features of the EyeMusic SSD, which gives the user whole-scene shape, location & color information. These features include higher resolution and attempts to overcome previous stumbling blocks by being freely available to download and run from a smartphone platform. We demonstrate with use the EyeMusic the potential of SSDs in noisy real-world scenarios for tasks such as identifying and manipulating objects. We then discuss the neural basis of using SSDs, and conclude by discussing other steps-in-progress on the path to making their practical use more widespread.

[1]  Shachar Maidenbaum,et al.  EyeMusic: Introducing a "visual" colorful experience for the blind using auditory sensory substitution. , 2014, Restorative neurology and neuroscience.

[2]  Peter B. L. Meijer,et al.  Visual experiences in the blind induced by an auditory sensory substitution device , 2010, Consciousness and Cognition.

[3]  Guido Bologna,et al.  Toward local and global perception modules for vision substitution , 2011, Neurocomputing.

[4]  F. Lepore,et al.  Sensory rehabilitation in the plastic brain. , 2011, Progress in brain research.

[5]  Amir Amedi,et al.  Visual Cortex Extrastriate Body-Selective Area Activation in Congenitally Blind People “Seeing” by Using Sounds , 2014, Current Biology.

[6]  Malika Auvray,et al.  The Vibe: a versatile vision-to-audition sensory substitution device , 2010 .

[7]  P. Sinha,et al.  Contribution of Color to Face Recognition , 2002, Perception.

[8]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[9]  Amir Amedi,et al.  Reading with Sounds: Sensory Substitution Selectively Activates the Visual Word Form Area in the Blind , 2012, Neuron.

[10]  Shachar Maidenbaum,et al.  Plasticity in Sensory Systems: Applying Plasticity to Visual Rehabilitation in Adulthood , 2012 .

[11]  C. Lenay,et al.  SENSORY SUBSTITUTION: LIMITS AND PERSPECTIVES , 2003 .

[12]  C. Trullemans,et al.  A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution , 1998, IEEE Transactions on Biomedical Engineering.

[13]  Ron Kupers,et al.  Tactile–‘visual’ acuity of the tongue in early blind individuals , 2007, Neuroreport.

[14]  R L Klatzky,et al.  Navigating without vision: basic and applied research. , 2001, Optometry and vision science : official publication of the American Academy of Optometry.

[15]  Shachar Maidenbaum,et al.  Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution , 2012, Scientific Reports.

[16]  A. Amedi,et al.  The brain as a flexible task machine: implications for visual rehabilitation using noninvasive vs. invasive approaches. , 2012, Current opinion in neurology.

[17]  Ron Kupers,et al.  Navigation with a sensory substitution device in congenitally blind individuals , 2011, Neuroreport.

[18]  Jung-Kyong Kim,et al.  Can you hear shapes you touch? , 2010, Experimental Brain Research.

[19]  Zachary Reynolds,et al.  Interactive Training for Sensory Substitution Devices , 2009 .

[20]  Thomas D. Wright,et al.  Where’s Wally? Audio–visual mismatch directs ocular saccades in sensory substitution , 2012 .

[21]  Shachar Maidenbaum,et al.  Author's Personal Copy Neuroscience and Biobehavioral Reviews Sensory Substitution: Closing the Gap between Basic Research and Widespread Practical Visual Rehabilitation Author's Personal Copy , 2022 .

[22]  S Levy-Tzedek,et al.  Fast, accurate reaching movements with a visual-to-auditory sensory substitution device. , 2012, Restorative neurology and neuroscience.