The Effectiveness of Visual and Audio Wayfinding Guidance on Smartglasses for People with Low Vision

Wayfinding is a critical but challenging task for people who have low vision, a visual impairment that falls short of blindness. Prior wayfinding systems for people with visual impairments focused on blind people, providing only audio and tactile feedback. Since people with low vision use their remaining vision, we sought to determine how audio feedback compares to visual feedback in a wayfinding task. We developed visual and audio wayfinding guidance on smartglasses based on de facto standard approaches for blind and sighted people and conducted a study with 16 low vision participants. We found that participants made fewer mistakes and experienced lower cognitive load with visual feedback. Moreover, participants with a full field of view completed the wayfinding tasks faster when using visual feedback. However, many participants preferred audio feedback because of its shorter learning curve. We propose design guidelines for wayfinding systems for low vision.

[1]  Kostas E. Bekris,et al.  Feasibility of interactive localization and navigation of people with visual impairments , 2010 .

[2]  Florence Gaunet,et al.  Verbal guidance rules for a localized wayfinding aid intended for blind-pedestrians in urban areas , 2006, Universal Access in the Information Society.

[3]  Ronald A. Rensink,et al.  Two strategies for learning a route in a driving simulator. , 1997 .

[4]  Saleh Alghamdi,et al.  Indoor navigational aid using active RFID and QR-code for sighted and blind people , 2013, 2013 IEEE Eighth International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[5]  Roberta L. Klatzky,et al.  Personal guidance system for the visually impaired , 1994, ASSETS.

[6]  Steven K. Feiner,et al.  Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system , 1999, Comput. Graph..

[7]  Roberta L. Klatzky,et al.  Nonvisual Route following with Guidance from a Simple Haptic or Auditory Display , 2007 .

[8]  Amy Hurst,et al.  "just let the cane hit it": how the blind and sighted see navigation differently , 2014, ASSETS.

[9]  Yuhang Zhao,et al.  Understanding Low Vision People's Visual Perception on Commercial Augmented Reality Glasses , 2017, CHI.

[10]  Shiang-Jiun Tsai,et al.  Risks of tinnitus, sensorineural hearing impairment, and sudden deafness in patients with non-migraine headache , 2019, PloS one.

[11]  Stephanie Ludi,et al.  Visually impaired orientation techniques in unfamiliar indoor environments: a user study , 2014, ASSETS.

[12]  Steven K. Feiner,et al.  Designing AR Visualizations to Facilitate Stair Navigation for People with Low Vision , 2019, UIST.

[13]  J. J. Higgins,et al.  The aligned rank transform for nonparametric factorial analyses using only anova procedures , 2011, CHI.

[14]  Xing-Dong Yang,et al.  An augmented reality sign-reading assistant for users with reduced vision , 2019, PloS one.

[15]  Alex Pentland,et al.  A wearable haptic navigation guidance system , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[16]  Constantine Stephanidis,et al.  Universal access in the information society , 1999, HCI.

[17]  Nikolaos G. Bourbakis,et al.  Wearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[18]  Richard A Armstrong,et al.  When to use the Bonferroni correction , 2014, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[19]  Abdelsalam Helal,et al.  Drishti: an integrated navigation system for visually impaired and disabled , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[20]  Anne L. Corn,et al.  Foundations of Low Vision: Clinical and Functional Perspectives , 2010 .

[21]  Amanda Stent,et al.  Iwalk: a lightweight navigation system for low-vision users , 2010, ASSETS '10.

[22]  Susan M. Downes,et al.  A Depth-Based Head-Mounted Visual Display to Aid Navigation in Partially Sighted Individuals , 2013, PloS one.

[23]  Meredith Ringel Morris,et al.  Use Cases and Impact of Audio-Based Virtual Exploration , 2020 .

[24]  Alois Ferscha,et al.  Augmented reality navigation systems , 2006, Universal Access in the Information Society.

[25]  Barry T. Thomas,et al.  Head-Mounted Mobility Aid for Low Vision Using Scene Classification Techniques , 1998, Int. J. Virtual Real..

[26]  Hironobu Takagi,et al.  NavCog3: An Evaluation of a Smartphone-Based Blind Indoor Navigation Assistant with Semantic Features in a Large-Scale Environment , 2017, ASSETS.

[27]  Hideo Makino,et al.  Development of navigation system for the blind using GPS and mobile phone combination , 1996, Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[28]  Zdenek Míkovec,et al.  Automatically generated landmark-enhanced navigation instructions for blind pedestrians , 2016, 2016 Federated Conference on Computer Science and Information Systems (FedCSIS).

[29]  Amy Hurst,et al.  "Pray before you step out": describing personal and situational blind navigation behaviors , 2013, ASSETS.

[30]  Sowmya Somanath,et al.  MRsive: An Augmented Reality Tool for Enhancing Wayfinding and Engagement with Art in Museums , 2019, HCI.

[31]  Tom Troscianko,et al.  Mobility performance of low‐vision adults using an electronic mobility aid , 2006, Clinical & experimental optometry.

[32]  Gordon E. Legge,et al.  Indoor Navigation by People with Visual Impairment Using a Digital Sign System , 2013, PloS one.

[33]  Robert M. Nishikawa,et al.  Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment , 2017 .

[34]  Alyssa Friend Wise,et al.  Getting Down to Details: Using Theories of Cognition and Learning to Inform Tangible User Interface Design , 2013, Interact. Comput..

[35]  Helen Petrie,et al.  Development of dialogue systems for a mobility aid for blind people: initial design and usability testing , 1996, Assets '96.

[36]  B. L. Bentzen,et al.  Audible Signage as a Wayfinding Aid: Verbal Landmark versus Talking Signs , 1995 .

[37]  Jeremy R. Cooperstock,et al.  Listen to it yourself!: evaluating usability of what's around me? for the blind , 2013, CHI.

[38]  Jizhong Xiao,et al.  Semantic Indoor Navigation with a Blind-User Oriented Augmented Reality , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[39]  Yuhang Zhao,et al.  CueSee: exploring visual cues for people with low vision to facilitate a visual search task , 2016, UbiComp.

[40]  Ryan M. Traylor,et al.  A Haptic Back Display for Attentional and Directional Cueing , 2003 .

[41]  Gert Jan Gelderblom,et al.  Users’ Evaluations of Four Electronic Travel Aids Aimed at Navigation for Persons who are Visually Impaired , 2011 .

[42]  Adam Rankin,et al.  Hologram stability evaluation for Microsoft HoloLens , 2017, Medical Imaging.

[43]  Roberta L. Klatzky,et al.  Evaluation of spatial displays for navigation without sight , 2006, TAP.

[44]  Kostas E. Bekris,et al.  Indoor Human Navigation Systems: A Survey , 2013, Interact. Comput..

[45]  James R. Marston,et al.  Cognitive load of navigating without vision when guided by virtual sound versus spatial language. , 2006, Journal of experimental psychology. Applied.

[46]  Roberta L. Klatzky,et al.  Assisting Wayfinding in Visually Impaired Travelers , 2006 .

[47]  Shyamanta M. Hazarika,et al.  IntelliNavi: Navigation for Blind Based on Kinect and Machine Learning , 2014, MIWAI.

[48]  Frederick Lippman,et al.  Blindness and vision impairment , 2008 .

[49]  Bernard C. Kress,et al.  11‐1: Invited Paper: Towards the Ultimate Mixed Reality Experience: HoloLens Display Architecture Choices , 2017 .

[50]  Hironobu Takagi,et al.  NavCog: a navigational cognitive assistant for the blind , 2016, MobileHCI.

[51]  J Faria,et al.  Electronic white cane for blind people navigation assistance , 2010, 2010 World Automation Congress.

[52]  Roberto Manduchi,et al.  Blind guidance using mobile computer vision: a usability study , 2010, ASSETS '10.

[53]  Christopher Kennard,et al.  Improving Mobility Performance in Low Vision With a Distance-Based Representation of the Visual Scene. , 2015, Investigative ophthalmology & visual science.

[54]  Gordon E. Legge,et al.  Digital Sign System for Indoor Wayfinding for the Visually Impaired , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[55]  I. Bailey,et al.  Visual Factors and Orientation‐Mobility Performance , 1982, American journal of optometry and physiological optics.

[56]  Atsuyuki Okabe,et al.  Wayfinding with a GPS-based mobile navigation system: A comparison with maps and direct experience , 2008 .

[57]  Yuhang Zhao,et al.  "It Looks Beautiful but Scary": How Low Vision People Navigate Stairs and Other Surface Level Changes , 2018, ASSETS.

[58]  T. Kuyk,et al.  Visual factors and mobility in persons with age-related macular degeneration. , 1999, Journal of rehabilitation research and development.

[59]  Yuhang Zhao,et al.  Finding a store, searching for a product: a study of daily challenges of low vision people , 2016, UbiComp.

[60]  Eelke Folmer,et al.  Headlock: a wearable navigation aid that helps blind cane users traverse large open spaces , 2014, ASSETS.

[61]  Ching-Tang Hsieh,et al.  An Indoor Obstacle Detection System Using Depth Information and Region Growth , 2015, Sensors.

[62]  Xing-Dong Yang,et al.  Using an Augmented Reality Device as a Distance-based Vision Aid—Promise and Limitations , 2018, Optometry and vision science : official publication of the American Academy of Optometry.

[63]  John Nicholson,et al.  RoboCart: toward robot-assisted navigation of grocery stores by the visually impaired , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[64]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[65]  Thomas Ertl,et al.  Design and development of an indoor navigation and object identification system for the blind , 2003, ASSETS.

[66]  Stephan Winter,et al.  Enriching Wayfinding Instructions with Local Landmarks , 2002, GIScience.

[67]  D. Geruschat,et al.  Low Vision Mobility Problems: Perceptions of O&M Specialists and Persons with Low Vision , 1992 .

[68]  Ken Sakamura,et al.  Navigating visually impaired travelers in a large train station using smartphone and bluetooth low energy , 2016, SAC.

[69]  Ariffin Abdul Mutalib,et al.  Smart Cane: Assistive Cane for Visually-impaired People , 2011, ArXiv.

[70]  Roberto Manduchi,et al.  Vibrotactile Guidance for Wayfinding of Blind Walkers , 2015, IEEE Transactions on Haptics.

[71]  Jan E Lovie-Kitchin,et al.  Visual Function, Visual Attention, and Mobility Performance in Low Vision , 2008, Optometry and vision science : official publication of the American Academy of Optometry.

[72]  Michel Denis,et al.  NAVIG: augmented reality guidance system for the visually impaired , 2012, Virtual Reality.

[73]  Roberto Manduchi,et al.  The last meter: blind visual guidance to a target , 2014, CHI.

[74]  Giovanni Fusco,et al.  Indoor Localization Using Computer Vision and Visual-Inertial Odometry , 2018, ICCHP.

[75]  H. Petrie,et al.  MOBIC: Designing a Travel Aid for Blind and Elderly People , 1996, Journal of Navigation.

[76]  A. Meule,et al.  Reporting and Interpreting Working Memory Performance in n-back Tasks , 2017, Front. Psychol..

[77]  Roberta L. Klatzky,et al.  Navigation System for the Blind: Auditory Display Modes and Guidance , 1998, Presence.

[78]  Richard E. Ladner,et al.  Smartphone haptic feedback for nonvisual wayfinding , 2011, ASSETS.

[79]  Vladimir A. Kulyukin,et al.  Robot-assisted shopping for the blind: issues in spatial cognition and product selection , 2008, Intell. Serv. Robotics.

[80]  Roberto Manduchi,et al.  Easy Return: An App for Indoor Backtracking Assistance , 2018, CHI.

[81]  R. Massof,et al.  Perceived visual ability for independent mobility in persons with retinitis pigmentosa. , 1999, Investigative ophthalmology & visual science.