Smartphone-based localization for blind navigation in building-scale indoor environments

Abstract Continuous, accurate, and real-time smartphone-based localization is a promising technology for supporting independent mobility of people with visual impairments. However, despite extensive research on indoor localization techniques, localization technologies are still not ready for deployment in large and complex environments such as shopping malls and hospitals, where navigation assistance is needed most. We identify six key challenges for accurate smartphone localization related to the large-scale nature of the navigation environments and the user’s mobility. To address these challenges, we present a series of techniques that enhance a probabilistic localization algorithm. The algorithm utilizes mobile device inertial sensors and Received Signal Strength (RSS) from Bluetooth Low Energy (BLE) beacons. We evaluate the proposed system in a 21,000 m 2 shopping mall that includes three multi-story buildings and a large open underground passageway. Experiments conducted in this environment demonstrate the effectiveness of the proposed technologies to improve localization accuracy. Field experiments with visually impaired participants confirm the practical performance of the proposed system in realistic use cases.

[1]  Chieko Asakawa,et al.  Environmental Factors in Indoor Navigation Based on Real-World Trajectories of Blind Users , 2018, CHI.

[2]  Roberto Manduchi,et al.  WeAllWalk , 2018, ACM Trans. Access. Comput..

[3]  Chieko Asakawa,et al.  How Context and User Behavior Affect Indoor Navigation Assistance for Blind People , 2018, W4A.

[4]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[5]  Hironobu Takagi,et al.  NavCog: turn-by-turn smartphone navigation assistant for people with visual impairments or blindness , 2016, W4A.

[6]  Robert Harle,et al.  Pedestrian localisation for indoor environments , 2008, UbiComp.

[7]  Chieko Asakawa,et al.  Turn Right: Analysis of Rotation Errors in Turn-by-Turn Navigation for Individuals with Visual Impairments , 2018, ASSETS.

[8]  Leor Banin WiFi FTM and Map Information Fusion for Accurate Positioning , 2016 .

[9]  Henry Tirri,et al.  A Probabilistic Approach to WLAN User Location Estimation , 2002, Int. J. Wirel. Inf. Networks.

[10]  Hao-Hua Chu,et al.  Unsupervised Learning for Solving RSS Hardware Variance Problem in WiFi Localization , 2009, Mob. Networks Appl..

[11]  Ken Sakamura,et al.  Navigating visually impaired travelers in a large train station using smartphone and bluetooth low energy , 2016, SAC.

[12]  Anshul Rai,et al.  Zee: zero-effort crowdsourcing for indoor localization , 2012, Mobicom '12.

[13]  Martin Magnusson,et al.  The three-dimensional normal-distributions transform : an efficient representation for registration, surface analysis, and loop detection , 2009 .

[14]  Masayuki Murata,et al.  Smartphone-based Indoor Localization for Blind Navigation across Building Complexes , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[15]  Swarun Kumar,et al.  Decimeter-Level Localization with a Single WiFi Access Point , 2016, NSDI.

[16]  Yunhao Liu,et al.  Locating in fingerprint space: wireless indoor localization with little human intervention , 2012, Mobicom '12.

[17]  G.B. Giannakis,et al.  Localization via ultra-wideband radios: a look at positioning aspects for future sensor networks , 2005, IEEE Signal Processing Magazine.

[18]  Kostas E. Bekris,et al.  Indoor Human Navigation Systems: A Survey , 2013, Interact. Comput..

[19]  Carl Posthuma,et al.  Crowdsourcing the Installation and Maintenance of Indoor Localization Infrastructure to Support Blind Navigation , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[20]  25 Blind Navigation and the Role of Technology , 2008 .

[21]  Ravi Jain,et al.  Error characteristics and calibration-free techniques for wireless LAN-based location estimation , 2004, MobiWac '04.

[22]  Guobin Shen,et al.  Experiencing and handling the diversity in data density and environmental locality in an indoor positioning service , 2014, MobiCom.

[23]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[24]  Paramvir Bahl,et al.  RADAR: an in-building RF-based user location and tracking system , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[25]  Lei Yu,et al.  Calibration-free fusion of step counter and wireless fingerprints for indoor localization , 2015, UbiComp.

[26]  Dieter Fox,et al.  Gaussian Processes for Signal Strength-Based Location Estimation , 2006, Robotics: Science and Systems.

[27]  Mingyan Liu,et al.  Mitigating Large Errors in WiFi-Based Indoor Localization for Smartphones , 2017, IEEE Transactions on Vehicular Technology.

[28]  Yunhao Liu,et al.  Enhancing wifi-based localization with visual clues , 2015, UbiComp.

[29]  Shinpei Kato,et al.  An Open Approach to Autonomous Vehicles , 2015, IEEE Micro.

[30]  Hironobu Takagi,et al.  NavCog: a navigational cognitive assistant for the blind , 2016, MobileHCI.

[31]  Bruno Sinopoli,et al.  ALPS: A Bluetooth and Ultrasound Platform for Mapping and Localization , 2015, SenSys.

[32]  Yong Ren,et al.  Does BTLE measure up against WiFi? A comparison of indoor location performance , 2014 .

[33]  Sheikh Tahir Bakhsh,et al.  Indoor positioning in Bluetooth networks using fingerprinting and lateration approach , 2011, 2011 International Conference on Information Science and Applications.

[34]  Hironobu Takagi,et al.  NavCog3: An Evaluation of a Smartphone-Based Blind Indoor Navigation Assistant with Semantic Features in a Large-Scale Environment , 2017, ASSETS.

[35]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[36]  Robert Harle,et al.  Location Fingerprinting With Bluetooth Low Energy Beacons , 2015, IEEE Journal on Selected Areas in Communications.

[37]  Agata Brajdic,et al.  Walk detection and step counting on unconstrained smartphones , 2013, UbiComp.

[38]  Stefano Chessa,et al.  A stigmergic approach to indoor localization using Bluetooth Low Energy beacons , 2015, 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[39]  Jan Peters,et al.  Local Gaussian process regression for real-time model-based robot control , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Eckehard Steinbach,et al.  Graph-based data fusion of pedometer and WiFi measurements for mobile indoor positioning , 2014, UbiComp.

[41]  Azeem J. Khan,et al.  Barometric phone sensors: more hype than hope! , 2014, HotMobile.

[42]  Vinod Namboodiri,et al.  GuideBeacon: Beacon-based indoor wayfinding for the blind, visually impaired, and disoriented , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[43]  Jue Wang,et al.  Dude, where's my card?: RFID positioning that works with multipath and non-line of sight , 2013, SIGCOMM.

[44]  Vijay Kumar,et al.  Online methods for radio signal mapping with mobile robots , 2010, 2010 IEEE International Conference on Robotics and Automation.

[45]  Jing Liu,et al.  Survey of Wireless Indoor Positioning Techniques and Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[46]  Yunhao Liu,et al.  From RSSI to CSI , 2013, ACM Comput. Surv..

[47]  Venkata N. Padmanabhan,et al.  Indoor localization without the pain , 2010, MobiCom.

[48]  Masayuki Murata,et al.  Achieving Practical and Accurate Indoor Navigation for People with Visual Impairments , 2017, W4A.

[49]  Shueng-Han Gary Chan,et al.  Wi-Fi Fingerprint-Based Indoor Positioning: Recent Advances and Comparisons , 2016, IEEE Communications Surveys & Tutorials.

[50]  Yunhao Liu,et al.  Mobility Increases Localizability , 2015, ACM Comput. Surv..

[51]  Chieko Asakawa,et al.  Variability in Reactions to Instructional Guidance during Smartphone-Based Assisted Navigation of Blind Users , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[52]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[53]  Robert Harle,et al.  A Survey of Indoor Inertial Positioning Systems for Pedestrians , 2013, IEEE Communications Surveys & Tutorials.

[54]  Jie Liu,et al.  A realistic evaluation and comparison of indoor location technologies: experiences and lessons learned , 2015, IPSN.

[55]  Feng Zhao,et al.  A reliable and accurate indoor localization method using phone inertial sensors , 2012, UbiComp.