Resource-efficient and Automated Image-based Indoor Localization

Image-based indoor localization has aroused much interest recently because it requires no infrastructure support. Previous approaches on image-based localization, due to their computation and storage requirements, often process queries at servers. This does not scale well, incurs round-trip delay, and requires constant network connectivity. Many also require users to manually confirm the shortlisted matched landmarks, which is inconvenient, slow, and prone to selection error. To overcome these limitations, we propose a highly automated (in terms of image confirmation after taking images) image-based localization algorithm (HAIL), distributed in mobile devices. HAIL achieves resource efficiency (in terms of storage and processing) by keeping only distinguishing visual features for each landmark, and employing the efficient k-d tree to search for features. It further utilizes motion sensors and map constraints to enhance the localization accuracy without user operation. We have implemented HAIL on Android platforms and conducted extensive experiments in a food plaza and a premium shopping mall. Experimental results show that it achieves much higher localization accuracy (reducing the localization error by more than 20%) and computation efficiency (by more than 40% in time) as compared with the state-of-the-art approaches.

[1]  Chadly Marouane,et al.  Indoor positioning using smartphone camera , 2011, 2011 International Conference on Indoor Positioning and Indoor Navigation.

[2]  Wen Hu,et al.  NaviGlass: Indoor Localisation Using Smart Glasses , 2016, EWSN.

[3]  Ting Zhu,et al.  Low-Overhead WiFi Fingerprinting , 2018, IEEE Transactions on Mobile Computing.

[4]  Jiang Dong,et al.  Indoor Tracking Using Crowdsourced Maps , 2016, 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[5]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[6]  Rodney X. Sturdivant,et al.  Applied Logistic Regression: Hosmer/Applied Logistic Regression , 2005 .

[7]  Noel E. O'Connor,et al.  Efficient Storage and Decoding of SURF Feature Points , 2012, MMM.

[8]  Zhong Liu,et al.  IONavi , 2017, ACM Trans. Sens. Networks.

[9]  Pei Zhang,et al.  Headio: zero-configured heading acquisition for indoor mobile devices through multimodal context sensing , 2013, UbiComp.

[10]  Wenbin Lin,et al.  Indoor Localization and Automatic Fingerprint Update with Altered AP Signals , 2017, IEEE Transactions on Mobile Computing.

[11]  Visa Koivunen,et al.  Cooperative Simultaneous Localization and Mapping by Exploiting Multipath Propagation , 2017, IEEE Transactions on Signal Processing.

[12]  Jue Wang,et al.  Dude, where's my card?: RFID positioning that works with multipath and non-line of sight , 2013, SIGCOMM.

[13]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[14]  Nicu Sebe,et al.  Knowing Where I Am: Exploiting Multi-Task Learning for Multi-view Indoor Image-based Localization , 2014, BMVC.

[15]  Xinbing Wang,et al.  HiQuadLoc: A RSS Fingerprinting Based Indoor Localization System for Quadrotors , 2017, IEEE Transactions on Mobile Computing.

[16]  François Chaumette,et al.  Appearance-Based Indoor Navigation by IBVS Using Line Segments , 2016, IEEE Robotics and Automation Letters.

[17]  Panu Turcot,et al.  Better matching with fewer features: The selection of useful features in large database recognition problems , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[18]  Nicu Sebe,et al.  Localize Me Anywhere, Anytime: A Multi-task Point-Retrieval Approach , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Ning Liu,et al.  Maxlifd: Joint Maximum Likelihood Localization Fusing Fingerprints and Mutual Distances , 2019, IEEE Transactions on Mobile Computing.

[20]  Baochun Li,et al.  $Tack:$ Learning Towards Contextual and Ephemeral Indoor Localization With Crowdsourcing , 2017, IEEE Journal on Selected Areas in Communications.

[21]  Yizhou Wang,et al.  Knitter: Fast, resilient single-user indoor floor plan construction , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[22]  Torsten Sattler,et al.  Are Large-Scale 3D Models Really Necessary for Accurate Visual Localization? , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Chenshu Wu,et al.  Gain Without Pain , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[24]  Matti Siekkinen,et al.  Exploring Vision-Based Techniques for Outdoor Positioning Systems: A Feasibility Study , 2017, IEEE Transactions on Mobile Computing.

[25]  Yunhao Liu,et al.  Indoor localization via multi-modal sensing on smartphones , 2016, UbiComp.

[26]  Torsten Sattler,et al.  Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Jie Liu,et al.  Indoor Localization Using FM Signals , 2013, IEEE Transactions on Mobile Computing.

[28]  Pan Hui,et al.  Ubii: Physical World Interaction Through Augmented Reality , 2017, IEEE Transactions on Mobile Computing.

[29]  Kaigui Bian,et al.  Sextant: Towards Ubiquitous Indoor Localization Service by Photo-Taking of the Environment , 2016, IEEE Transactions on Mobile Computing.

[30]  Masatoshi Okutomi,et al.  Visual Place Recognition with Repetitive Structures , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Sen Wang,et al.  Poster Abstract: Efficient Visual Positioning with Adaptive Parameter Learning , 2016, 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[32]  Fei Gu,et al.  WAIPO: A Fusion-Based Collaborative Indoor Localization System on Smartphones , 2017, IEEE/ACM Transactions on Networking.

[33]  Lei Deng,et al.  Incremental image set querying based localization , 2016, Neurocomputing.

[34]  David G. Lowe,et al.  Scalable Nearest Neighbor Algorithms for High Dimensional Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Claudio Piciarelli,et al.  Visual Indoor Localization in Known Environments , 2016, IEEE Signal Processing Letters.

[36]  Moustafa Youssef,et al.  A Fine-Grained Indoor Location-Based Social Network , 2017, IEEE Transactions on Mobile Computing.

[37]  Muhamad Risqi U. Saputra,et al.  Visual SLAM and Structure from Motion in Dynamic Environments , 2018, ACM Comput. Surv..

[38]  Yunhao Liu,et al.  Enhancing wifi-based localization with visual clues , 2015, UbiComp.

[39]  Yan Yan,et al.  A Fast 3D Indoor-Localization Approach Based on Video Queries , 2016, MMM.

[40]  Luigi di Stefano,et al.  On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Tomás Pajdla,et al.  Avoiding Confusing Features in Place Recognition , 2010, ECCV.

[42]  Cordelia Schmid,et al.  Improving Bag-of-Features for Large Scale Image Search , 2010, International Journal of Computer Vision.

[43]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[44]  Xinbing Wang,et al.  Improve Accuracy of Fingerprinting Localization with Temporal Correlation of the RSS , 2018, IEEE Transactions on Mobile Computing.

[45]  Eckehard G. Steinbach,et al.  Camera-based indoor positioning using scalable streaming of compressed binary image signatures , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[46]  Ning Liu,et al.  SLAC: Calibration-Free Pedometer-Fingerprint Fusion for Indoor Localization , 2018, IEEE Transactions on Mobile Computing.

[47]  Mun Choon Chan,et al.  MPiLoc: Self-Calibrating Multi-Floor Indoor Localization Exploiting Participatory Sensing , 2018, IEEE Transactions on Mobile Computing.

[48]  Binoy Pinto,et al.  Speeded Up Robust Features , 2011 .

[49]  Qi Han,et al.  The Emergence of Visual Crowdsensing: Challenges and Opportunities , 2017, IEEE Communications Surveys & Tutorials.

[50]  Nicu Sebe,et al.  Memory efficient large-scale image-based localization , 2014, Multimedia Tools and Applications.

[51]  Daniel P. Huttenlocher,et al.  Location Recognition Using Prioritized Feature Matching , 2010, ECCV.

[52]  Mani Golparvar-Fard,et al.  Fast and scalable structure-from-motion based localization for high-precision mobile augmented reality systems , 2016, mUX: The Journal of Mobile User Experience.

[53]  Chenshu Wu,et al.  Automatic Radio Map Adaptation for Indoor Localization Using Smartphones , 2018, IEEE Transactions on Mobile Computing.

[54]  Mo Li,et al.  Travi-Navi: self-deployable indoor navigation system , 2014, MobiCom.

[55]  Torsten Sattler,et al.  Improving Image-Based Localization by Active Correspondence Search , 2012, ECCV.