Integrated online localization and navigation for people with visual impairments using smart phones

Indoor localization and navigation systems for individuals with visual impairments (VI) typically rely upon extensive augmentation of the physical space or heavy, expensive sensors; thus, few systems have been adopted. This work describes a system able to guide people with VI through buildings using inexpensive sensors, such as accelerometers, which are available in portable devices like smart phones. The method takes advantage of feedback from the human user, who confirms the presence of landmarks. The system calculates the user's location in real time and uses it to provide audio instructions on how to reach the desired destination. Previous work suggested that the accuracy of the approach depended on the type of directions and the availability of an appropriate transition model for the user. A critical parameter for the transition model is the user's step length. The current work investigates different schemes for automatically computing the user's step length and reducing the dependency of the approach to the definition of an accurate transition model. Furthermore, the direction provision method is able to use the localization estimate and adapt to failed executions of paths by the users. Experiments are presented that evaluate the accuracy of the overall integrated system, which is executed online on a smart phone. Both people with visual impairments, as well as blindfolded sighted people, participated in the experiments. The experiments included paths along multiple floors, that required the use of stairs and elevators.

[1]  David A. Ross,et al.  Development of a Wearable Computer Orientation System , 2002, Personal and Ubiquitous Computing.

[2]  Joel A. Hesch,et al.  A 3D pose estimator for the visually impaired , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Jean-Paul Laumond,et al.  An Optimality Principle Governing Human Walking , 2008, IEEE Transactions on Robotics.

[4]  Michitaka Hirose,et al.  Virtual leading blocks for the deaf-blind: a real-time way-finder by verbal-nonverbal hybrid interface and high-density RFID tag space , 2004, IEEE Virtual Reality 2004.

[5]  Thomas Ertl,et al.  Design and development of an indoor navigation and object identification system for the blind , 2003, ASSETS.

[6]  Wolfram Burgard,et al.  Robust Monte Carlo localization for mobile robots , 2001, Artif. Intell..

[7]  Peter C. Cheeseman,et al.  Estimating uncertain spatial relationships in robotics , 1986, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[8]  Seth J. Teller,et al.  Wide-Area Egomotion Estimation from Known 3D Structure , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Gordon E Legge,et al.  Learning Building Layouts with Non-Geometric Visual Information: The Effects of Visual Impairment and Age , 2008, Perception.

[10]  Gordon E. Legge,et al.  Digital Sign System for Indoor Wayfinding for the Visually Impaired , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[11]  Kostas E. Bekris,et al.  Feasibility of interactive localization and navigation of people with visual impairments , 2010 .

[12]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[13]  Manuela M. Veloso,et al.  Sensor resetting localization for poorly modelled mobile robots , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[14]  Kostas E. Bekris,et al.  On the feasibility of using wireless ethernet for indoor localization , 2004, IEEE Transactions on Robotics and Automation.

[15]  Tomohiro Amemiya,et al.  Orienting Kinesthetically: A Haptic Handheld Wayfinder for People with Visual Impairments , 2010, TACC.

[16]  Akio Yamamoto,et al.  Virtual environment to evaluate multimodal feedback strategies for augmented navigation of the visually impaired , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[17]  Ken Sakamura,et al.  Assisting mobility of the disabled using space-identifying ubiquitous infrastructure , 2008, Assets '08.

[18]  Shraga Shoval,et al.  Auditory guidance with the Navbelt-a computerized travel aid for the blind , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[19]  Frank Dellaert,et al.  A Multi-Camera Pose Tracker for Assisting the Visually Impaired , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[20]  Abdelsalam Helal,et al.  Drishti: an integrated indoor/outdoor blind navigation system and service , 2004, Second IEEE Annual Conference on Pervasive Computing and Communications, 2004. Proceedings of the.

[21]  X. R. Li,et al.  Multiple-model estimation with variable structure. III. Model-group switching algorithm , 1999 .

[22]  Frank Dellaert,et al.  Map-based priors for localization , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[23]  N E Manos,et al.  Stochastic Models , 1960, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[24]  J. Lovie-Kitchin,et al.  Mobility performance with retinitis pigmentosa , 1997 .

[25]  Roberta L. Klatzky,et al.  Navigation System for the Blind: Auditory Display Modes and Guidance , 1998, Presence.

[26]  Kostas E. Bekris,et al.  Robotics-Based Location Sensing Using Wireless Ethernet , 2002, MobiCom '02.

[27]  Quentin Ladetto,et al.  Combining Gyroscopes, Magnetic Compass and GPS for Pedestrian Navigation , 2001 .

[28]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[29]  David Mioduser,et al.  Multisensory Virtual Environment for Supporting Blind Persons' Acquisition of Spatial Cognitive Mapping – a Case Study , 2001 .

[30]  Amir Averbuch,et al.  Interacting Multiple Model Methods in Target Tracking: A Survey , 1988 .

[31]  Avi Parush,et al.  Landmarks for navigators who are visually impaired , 2005 .

[32]  M. Bernardine Dias,et al.  Robust Indoor Localization on a Commercial Smart Phone , 2012, ANT/MobiWIS.

[33]  F. Groen,et al.  Human walking estimation with radar , 2003 .

[34]  John Nicholson,et al.  RFID in robot-assisted indoor navigation for the visually impaired , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[35]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[36]  Peter S. Maybeck,et al.  Stochastic Models, Estimation And Control , 2012 .

[37]  T. H. Riehle,et al.  An indoor navigation system to support the visually impaired , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[38]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[39]  Vincent Kanade,et al.  Clustering Algorithms , 2021, Wireless RF Energy Transfer in the Massive IoT Era.

[40]  Kostas E. Bekris,et al.  Integrated online localization and navigation for people with visual impairments using smart phones , 2012, ICRA.

[41]  Joel A. Hesch,et al.  An Indoor Localization Aid for the Visually Impaired , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[42]  Angelo M. Sabatini,et al.  A step toward GPS/INS personal navigation systems: real-time assessment of gait by foot inertial sensing , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[43]  Kostas E. Bekris,et al.  The user as a sensor: navigating users with visual impairments in indoor spaces using tactile landmarks , 2012, CHI.

[44]  Roberto Manduchi,et al.  Dynamic environment exploration using a virtual white cane , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).