VILL: Towards Efficient and Automatic Visual Landmark Labeling

Of all indoor localization techniques, vision-based localization emerges as a promising one, mainly due to the ubiquity of rich visual features. Visual landmarks, which present distinguishing textures, play a fundamental role in visual indoor localization. However, few researches focus on visual landmark labeling. Preliminary arts usually designate a surveyor to select and record visual landmarks, which is tedious and time-consuming. Furthermore, due to structural changes (e.g., renovation), the visual landmark database may be outdated, leading to degraded localization accuracy. To overcome these limitations, we propose VILL, a user-friendly, efficient and accurate approach for visual landmark labeling. VILL asks a user to sweep the camera to take a video clip of his/her surroundings. In the construction stage, VILL identifies unlabeled visual landmarks from videos adaptively according to the graph-based visual correlation representation. Based on the spatial correlations between selected anchor landmarks, VILL estimates locations of unlabeled ones on the floorplan accurately. In the update stage, VILL formulates an alteration identification model based on the judgements from different users to identify altered landmarks accurately. Extensive experimental results in two different trial sites show that VILL reduces the site survey substantially (by at least 65.9%) and achieves comparable accuracy.

[1]  Qi Han,et al.  Acceptance-Aware Mobile Crowdsourcing Worker Recruitment in Social Networks , 2021, IEEE Transactions on Mobile Computing.

[2]  Ruipeng Gao,et al.  Unsupervised Learning of Monocular Depth and Ego-Motion in Outdoor/Indoor Environments , 2022, IEEE Internet of Things Journal.

[3]  Xin Pei,et al.  Self-Supervised 3D Reconstruction and Ego-Motion Estimation Via On-Board Monocular Video , 2022, IEEE Transactions on Intelligent Transportation Systems.

[4]  Yonghe Liu,et al.  Compressive Sensing Based Distributed Data Storage for Mobile Crowdsensing , 2022, ACM Trans. Sens. Networks.

[5]  Saewoong Bahk,et al.  Smartphone Based Indoor Path Estimation and Localization Without Human Intervention , 2022, IEEE Transactions on Mobile Computing.

[6]  Xingfa Shen,et al.  Transition Model–driven Unsupervised Localization Framework Based on Crowd-sensed Trajectory Data , 2022, ACM Trans. Sens. Networks.

[7]  Landu Jiang,et al.  Beyond RSS: A PRR and SNR Aided Localization System for Transceiver-Free Target in Sparse Wireless Networks , 2021, IEEE Transactions on Mobile Computing.

[8]  Alp Eren Sari,et al.  SelfVIO: Self-Supervised Deep Monocular Visual-Inertial Odometry and Depth Estimation , 2019, Neural Networks.

[9]  Jun Yu Li,et al.  Joint Optimization of Depth and Ego-Motion for Intelligent Autonomous Vehicles , 2022, IEEE Transactions on Intelligent Transportation Systems.

[10]  WuHang,et al.  Pedometer-free Geomagnetic Fingerprinting with Casual Walking Speed , 2022, ACM Trans. Sens. Networks.

[11]  D LaneNicholas,et al.  Wireless Localization with Spatial-Temporal Robust Fingerprints , 2021, ACM Trans. Sens. Networks.

[12]  Fang Liu,et al.  The Crowd Wisdom for Location Privacy of Crowdsensing Photos , 2021, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[13]  Hongbo Jiang,et al.  Fly-Navi: A Novel Indoor Navigation System With On-the-Fly Map Generation , 2021, IEEE Transactions on Mobile Computing.

[14]  Xinglin Zhang,et al.  Price Learning-based Incentive Mechanism for Mobile Crowd Sensing , 2021, ACM Trans. Sens. Networks.

[15]  Zheng Yang,et al.  Train Once, Locate Anytime for Anyone: Adversarial Learning based Wireless Localization , 2021, IEEE INFOCOM 2021 - IEEE Conference on Computer Communications.

[16]  Chenshu Wu,et al.  Smartphone-Based Indoor Visual Navigation with Leader-Follower Mode , 2021, ACM Trans. Sens. Networks.

[17]  Jianwei Huang,et al.  Strategic Information Revelation in Crowdsourcing Systems Without Verification , 2021, IEEE INFOCOM 2021 - IEEE Conference on Computer Communications.

[18]  Nazanin Rahnavard,et al.  DyLoc: Dynamic Localization for Massive MIMO Using Predictive Recurrent Neural Networks , 2021, IEEE INFOCOM 2021 - IEEE Conference on Computer Communications.

[19]  Fabio A. M. Cappabianco,et al.  Autonomous Visual Navigation for Mobile Robots , 2020, ACM Comput. Surv..

[20]  Meng-Shiuan Pan,et al.  ezNavi: An Easy-to-Operate Indoor Navigation System Based on Pedestrian Dead Reckoning and Crowdsourced User Trajectories , 2019, IEEE Transactions on Mobile Computing.

[21]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Qiang Ma,et al.  Enabling Surveillance Cameras to Navigate , 2020, 2020 29th International Conference on Computer Communications and Networks (ICCCN).

[23]  Pascal Fua,et al.  Local Non-Rigid Structure-From-Motion From Diffeomorphic Mappings , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Rui Zhang,et al.  IndoorWaze: A Crowdsourcing-Based Context-Aware Indoor Navigation System , 2020, IEEE Transactions on Wireless Communications.

[25]  Christos Laoudias,et al.  Image and WLAN Bimodal Integration for Indoor User Localization , 2020, IEEE Transactions on Mobile Computing.

[26]  Ravi Kuber,et al.  Towards More Universal Wayfinding Technologies: Navigation Preferences Across Disabilities , 2020, CHI.

[27]  Keiko Katsuragawa,et al.  Understanding Viewport- and World-based Pointing with Everyday Smart Devices in Immersive Augmented Reality , 2020, CHI.

[28]  Dinesh Bharadia,et al.  Deep learning based wireless localization for indoor navigation , 2020, MobiCom.

[29]  Fan Zhang,et al.  WiFi and Vision-Integrated Fingerprint for Smartphone-Based Self-Localization in Public Indoor Scenes , 2020, IEEE Internet of Things Journal.

[30]  Jie Hao,et al.  Enhancing Camera-Based Multimodal Indoor Localization With Device-Free Movement Measurement Using WiFi , 2020, IEEE Internet of Things Journal.

[31]  Yinda Zhang,et al.  DeepSFM: Structure From Motion Via Deep Bundle Adjustment , 2019, ECCV.

[32]  Zhichao Cao,et al.  EyeLoc: Smartphone Vision Enabled Plug-n-play Indoor Localization in Large Shopping Malls , 2019, 2019 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN).

[33]  Shahrokh Valaee,et al.  Indoor Localization Improved by Spatial Context—A Survey , 2019, ACM Comput. Surv..

[34]  Antti Ylä-Jääski,et al.  ViNav: A Vision-Based Indoor Navigation System for Smartphones , 2019, IEEE Transactions on Mobile Computing.

[35]  Bing Zhou,et al.  Fast and Resilient Indoor Floor Plan Construction with a Single User , 2019, IEEE Transactions on Mobile Computing.

[36]  Yunhao Liu,et al.  Pair-Navi: Peer-to-Peer Indoor Navigation with Mobile Visual SLAM , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[37]  Jiashi Feng,et al.  Few-Shot Adaptive Faster R-CNN , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Xiaonan Luo,et al.  Resource-efficient and Automated Image-based Indoor Localization , 2019, ACM Trans. Sens. Networks.

[39]  Xin Wang,et al.  Few-Shot Object Detection via Feature Reweighting , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[40]  Yasin Almalioglu,et al.  GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[41]  Zhong Liu,et al.  SISE: Self-Updating of Indoor Semantic Floorplans for General Entities , 2018, IEEE Transactions on Mobile Computing.

[42]  D. Fritsch,et al.  ComNSense: Grammar-Driven Crowd-Sourcing of Point Clouds for Automatic Indoor Mapping , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[43]  Qingquan Li,et al.  Visual landmark sequence-based indoor localization , 2017, GeoAI@SIGSPATIAL.

[44]  Moustafa Youssef,et al.  Automatic Rich Map Semantics Identification Through Smartphone-Based Crowd-Sensing , 2017, IEEE Transactions on Mobile Computing.

[45]  Yunhao Liu,et al.  Indoor localization via multi-modal sensing on smartphones , 2016, UbiComp.

[46]  Kaigui Bian,et al.  Multi-Story Indoor Floor Plan Reconstruction via Mobile Crowdsensing , 2016, IEEE Transactions on Mobile Computing.

[47]  Kaigui Bian,et al.  Sextant: Towards Ubiquitous Indoor Localization Service by Photo-Taking of the Environment , 2016, IEEE Transactions on Mobile Computing.

[48]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[50]  Kari Sentz,et al.  Combination of Evidence in Dempster-Shafer Theory , 2002 .

[51]  Lotfi A. Zadeh,et al.  A Simple View of the Dempster-Shafer Theory of Evidence and Its Implication for the Rule of Combination , 1985, AI Mag..