Depth-To-Audio Sensory Substitution for Increasing the Accessibility of Virtual Environments

As most computerized information is visual, it is not directly accessible to the blind and visually impaired. This challenge is especially great when discussing graphical virtual environments. This is especially unfortunate as such environments hold great potential for the blind community for uses such as social interaction, online education and especially for safe mobility training from the safety and comfort of their home. While several previous attempts have increased the accessibility of these environments current tools are still far from making them properly accessible.We suggest the use of Sensory Substitution Devices (SSDs) as another step in increasing the accessibility of such environments by offering the user more raw "visual" information about the scene via other senses. Specifically, we explore here the use of a minimal-SSD based upon the EyeCane, which uses point depth distance information of a single pixel, for tasks such as virtual shape recognition and virtual navigation. We show both success and the fast learned use of this transformation by our users in these tasks, demonstrating the potential for this approach and end with a call for its addition to accessibility toolboxes.

[1]  Vicki L. Hanson,et al.  PowerUp: an accessible virtual world , 2008, Assets '08.

[2]  Jaime Sánchez,et al.  Teaching the Blind to Find Their Way by Playing Video Games , 2012, PloS one.

[3]  Jonathan A. Stirk,et al.  Spatial information transfer from virtual to real versions of the Kiel locomotor maze , 2000, Behavioural Brain Research.

[4]  Laurence R. Harris,et al.  Plasticity in Sensory Systems , 2012 .

[5]  Ron Kupers,et al.  Tactile–‘visual’ acuity of the tongue in early blind individuals , 2007, Neuroreport.

[6]  Orly Lahav,et al.  Improving Orientation and Mobility Skills through Virtual Environments for People Who Are Blind: Past Research and Future Potential , 2014 .

[7]  Nicholas A. Giudice,et al.  Representing 3D Space in Working Memory: Spatial Images from Vision, Hearing, Touch, and Language , 2013 .

[8]  Shachar Maidenbaum,et al.  Increasing Accessibility to the Blind of Virtual Environments, Using a Virtual Mobility Aid Based On the "EyeCane": Feasibility Study , 2013, PloS one.

[9]  A. Amedi,et al.  The brain as a flexible task machine: implications for visual rehabilitation using noninvasive vs. invasive approaches. , 2012, Current opinion in neurology.

[10]  David Mioduser,et al.  Exploration of Unknown Spaces by People Who are Blind Using a Multi-Sensory Virtual Environment , 2004 .

[11]  S Levy-Tzedek,et al.  Virtual 3 D shape and orientation discrimination using point distance information , 2012 .

[12]  John H. Bailey,et al.  Virtual spaces and real world places: transfer of route knowledge , 1996, Int. J. Hum. Comput. Stud..

[13]  Geraldine Fitzpatrick,et al.  Toward accessible 3D virtual environments for the blind and visually impaired , 2008, DIMEA.

[14]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[15]  Shachar Maidenbaum,et al.  Plasticity in Sensory Systems: Applying Plasticity to Visual Rehabilitation in Adulthood , 2012 .

[16]  R. Manduchi,et al.  Mobility-Related Accidents Experienced by People with Visual Impairment , 2010 .

[17]  Ron Kupers,et al.  Navigation with a sensory substitution device in congenitally blind individuals , 2011, Neuroreport.

[18]  Amir Amedi,et al.  ‘Visual’ Acuity of the Congenitally Blind Using Visual-to-Auditory Sensory Substitution , 2012, PloS one.

[19]  Shachar Maidenbaum,et al.  Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution , 2012, Scientific Reports.

[20]  Shachar Maidenbaum,et al.  EyeMusic: Introducing a "visual" colorful experience for the blind using auditory sensory substitution. , 2014, Restorative neurology and neuroscience.