Exploring Virtual Environments by Visually Impaired Using a Mixed Reality Cane Without Visual Feedback

Though virtual reality (VR) has been advanced to certain levels of maturity in recent years, the general public, especially the population of the blind and visually impaired (BVI), still cannot enjoy the benefit provided by VR. Current VR accessibility applications have been developed either on expensive head-mounted displays or with extra accessories and mechanisms, which are either not accessible or inconvenient for BVI individuals. In this paper, we present a mobile VR app that enables BVI users to access a virtual environment on an iPhone in order to build their skills of perception and recognition of the virtual environment and the virtual objects in the environment. The app uses the iPhone on a selfie stick to simulate a long cane in VR, and applies Augmented Reality (AR) techniques to track the iPhone’s real-time poses in an empty space of the real world, which is then synchronized to the long cane in the VR environment. Due to the use of mixed reality (the integration of VR & AR), we call it the Mixed Reality cane (MR Cane), which provides BVI users auditory and vibrotactile feedback whenever the virtual cane comes in contact with objects in VR. Thus, the MR Cane allows BVI individuals to interact with the virtual objects and identify approximate sizes and locations of the objects in the virtual environment. We performed preliminary user studies with blind-folded participants to investigate the effectiveness of the proposed mobile approach and the results indicate that the proposed MR Cane could be effective to help BVI individuals in understanding the interaction with virtual objects and exploring 3D virtual environments. The MR Cane concept can be extended to new applications of navigation, training and entertainment for BVI individuals without more significant efforts.

[1]  Geraldine Fitzpatrick,et al.  Toward accessible 3D virtual environments for the blind and visually impaired , 2008, DIMEA.

[2]  Devender R. Banda,et al.  A Survey on the Use of Mobile Applications for People who Are Visually Impaired , 2017 .

[3]  J. Rieser,et al.  How Persons with Visual Impairments Explore Novel Spaces: Strategies of Good and Poor Performers , 1993 .

[4]  Timo Götzelmann,et al.  FeelVR: Haptic Exploration of Virtual Objects , 2018, PETRA.

[5]  Amy Hurst,et al.  "Pray before you step out": describing personal and situational blind navigation behaviors , 2013, ASSETS.

[6]  David J. Brown,et al.  An interface to virtual environments for people who are blind using Wii technology ‐ mental models and navigation , 2009 .

[7]  Elke E. Mattheiss,et al.  Mobile location-based games to support orientation & mobility training for visually impaired students , 2018, MobileHCI.

[8]  Timo Götzelmann,et al.  Real World VR Proxies to Support Blind People in Mobility Training , 2018, Mensch & Computer Workshopband.

[9]  Amy Hurst,et al.  "just let the cane hit it": how the blind and sighted see navigation differently , 2014, ASSETS.

[10]  M. Strintzis,et al.  CyberGrasp and PHANTOM Integration : Enhanced Haptic Access for Visually Impaired Users , 2004 .

[11]  Zhigang Zhu,et al.  Indoor Map Learning for the Visually Impaired , 2017 .

[12]  Edward Cutrell,et al.  Virtual Reality Without Vision: A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds , 2020, CHI.

[13]  Meredith Ringel Morris,et al.  SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision , 2019, CHI.

[14]  Timo Götzelmann,et al.  First Steps Towards Walk-In-Place Locomotion and Haptic Feedback in Virtual Reality for Visually Impaired , 2019, CHI Extended Abstracts.

[15]  Michel Denis,et al.  Exploration of architectural spaces by blind people using auditory virtual reality for the construction of spatial knowledge , 2014, Int. J. Hum. Comput. Stud..

[16]  Orly Lahav,et al.  A Virtual Environment for People Who Are Blind - A Usability Study. , 2012, Journal of assistive technologies.

[17]  Anatole Lécuyer,et al.  HOMERE: a multimodal system for visually impaired people to explore virtual environments , 2003, IEEE Virtual Reality, 2003. Proceedings..

[18]  Meredith Ringel Morris,et al.  Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation , 2018, CHI.

[19]  Timo Götzelmann,et al.  Evaluation of different types of haptic feedback influencing the task-based presence and performance in virtual reality , 2019, PETRA.