Zero-Shot Multi-View Indoor Localization via Graph Location Networks

Indoor localization is a fundamental problem in location-based applications. Current approaches to this problem typically rely on Radio Frequency technology, which requires not only supporting infrastructures but human efforts to measure and calibrate the signal. Moreover, data collection for all locations is indispensable in existing methods, which in turn hinders their large-scale deployment. In this paper, we propose a novel neural network based architecture Graph Location Networks (GLN) to perform infrastructure-free, multi-view image based indoor localization. GLN makes location predictions based on robust location representations extracted from images through message-passing networks. Furthermore, we introduce a novel zero-shot indoor localization setting and tackle it by extending the proposed GLN to a dedicated zero-shot version, which exploits a novel mechanism Map2Vec to train location-aware embeddings and make predictions on novel unseen locations. Our extensive experiments show that the proposed approach outperforms state-of-the-art methods in the standard setting, and achieves promising accuracy even in the zero-shot setting where data for half of the locations are not available. The source code and datasets are publicly available.

[1]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[2]  Alex Fout,et al.  Protein Interface Prediction using Graph Convolutional Networks , 2017, NIPS.

[3]  Christoph H. Lampert,et al.  Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[5]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[6]  Liviu Iftode,et al.  Indoor Localization Using Camera Phones , 2006, Seventh IEEE Workshop on Mobile Computing Systems & Applications (WMCSA'06 Supplement).

[7]  Ig-Jae Kim,et al.  Indoor location sensing using geo-magnetism , 2011, MobiSys '11.

[8]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[9]  Kavita Bala,et al.  Learning visual similarity for product design with convolutional neural networks , 2015, ACM Trans. Graph..

[10]  Jim Sinopoli,et al.  Smart Buildings , 2017, Encyclopedia of GIS.

[11]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[12]  Bernt Schiele,et al.  Evaluation of output embeddings for fine-grained image classification , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[14]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[15]  Sverre Holm,et al.  Hybrid ultrasound-RFID indoor positioning: Combining the best of both worlds , 2009, 2009 IEEE International Conference on RFID.

[16]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[17]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[18]  Haiping Lu,et al.  Graph Node-Feature Convolution for Representation Learning , 2018, ArXiv.

[19]  Roland Siegwart,et al.  Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization , 2018, CoRL.

[20]  Roger Zimmermann,et al.  Multi-Level Fusion based Class-aware Attention Model for Weakly Labeled Audio Tagging , 2019, ACM Multimedia.

[21]  Tao Mei,et al.  POINet: Pose-Guided Ovonic Insight Network for Multi-Person Pose Tracking , 2019, ACM Multimedia.

[22]  Peter I. Corke,et al.  Visual Place Recognition: A Survey , 2016, IEEE Transactions on Robotics.

[23]  Jong-Suk Choi,et al.  Advanced indoor localization using ultrasonic sensor and digital compass , 2008, 2008 International Conference on Control, Automation and Systems.

[24]  Kaigui Bian,et al.  Sextant: Towards Ubiquitous Indoor Localization Service by Photo-Taking of the Environment , 2016, IEEE Transactions on Mobile Computing.

[25]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Tomás Pajdla,et al.  NetVLAD: CNN Architecture for Weakly Supervised Place Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Swarun Kumar,et al.  Decimeter-Level Localization with a Single WiFi Access Point , 2016, NSDI.

[30]  Chunyan Miao,et al.  A Survey of Zero-Shot Learning , 2019, ACM Trans. Intell. Syst. Technol..

[31]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[32]  Bernt Schiele,et al.  Zero-Shot Learning — The Good, the Bad and the Ugly , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[34]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[36]  Luming Zhang,et al.  Multiview and Multimodal Pervasive Indoor Localization , 2017, ACM Multimedia.

[37]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[38]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[39]  Christoph H. Lampert,et al.  Attribute-Based Classification for Zero-Shot Visual Object Categorization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Deborah Snoonian,et al.  Control systems: smart buildings , 2003 .

[41]  Lise Getoor,et al.  Collective Classi!cation in Network Data , 2008 .

[42]  Hesham Mostafa,et al.  Permutohedral-GCN: Graph Convolutional Networks with Global Attention , 2020, ArXiv.

[43]  Avideh Zakhor,et al.  Image Based Localization in Indoor Environments , 2013, 2013 Fourth International Conference on Computing for Geospatial Research and Application.

[44]  Yuji Matsumoto,et al.  Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach , 2017, ArXiv.

[45]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[46]  Jiří Kárník,et al.  Summary of available indoor location techniques , 2016 .

[47]  Raia Hadsell,et al.  Graph networks as learnable physics engines for inference and control , 2018, ICML.

[48]  Niko Sünderhauf,et al.  On the performance of ConvNet features for place recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[49]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[50]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Kenichi Mase,et al.  Improved Indoor Location Estimation Using Fluorescent Light Communication System with a Nine-Channel Receiver , 2010, IEICE Trans. Commun..

[52]  Philip H. S. Torr,et al.  An embarrassingly simple approach to zero-shot learning , 2015, ICML.

[53]  Selim Aksoy,et al.  Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Guobin Shen,et al.  Magicol: Indoor Localization Using Pervasive Magnetic Field and Opportunistic WiFi Sensing , 2015, IEEE Journal on Selected Areas in Communications.

[55]  Feng Zhao,et al.  A reliable and accurate indoor localization method using phone inertial sensors , 2012, UbiComp.