SISE: Self-Updating of Indoor Semantic Floorplans for General Entities

Indoor semantic floorplan is important for a range of location based service (LBS) applications, attracting many research efforts in several years. In many cases, the out-of-date indoor semantic floorplans would gradually deteriorate and even break down the LBS performance. Thus, it is important to automatically update changed semantics of indoor floorplans caused by environmental variation. However, few research has been focused on the continuous semantic updating problem. This paper presents SISE as a mobile crowdsourcing system that uses a new abstraction for indoor general entities and their semantics, enGraph, to automatically update changed semantics of indoor floorplans using images and inertial data. We first propose efficient methods to generate enGraph. Thus, an image can be associated with an indoor semantic floorplan. Accordingly, we formulate the enGraph matching problem and then propose a quality-based maximum common subgraph matching algorithm so that entities extracted from an image can be corresponded to entities in the indoor semantic floorplan. Furthermore, we propose a quadrant comparison algorithm and a region shrink based localization algorithm to detect and localize changed entities. Thus, the new semantics can be labeled and out-of-date semantics can be removed. Extensive experiments have been conducted on real and synthetic data. Experimental results show that 80 percent of out-of-date semantics of indoor general entities can be updated by SISE.

[1]  Moustafa Youssef,et al.  SemSense: Automatic construction of semantic indoor floorplans , 2015, 2015 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[2]  Andrew Zisserman,et al.  Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition , 2014, ArXiv.

[3]  Jignesh M. Patel,et al.  SAGA: a subgraph matching tool for biological graphs , 2007, Bioinform..

[4]  Sanja Fidler,et al.  Lost Shopping! Monocular Localization in Large Indoor Spaces , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Zhong Liu,et al.  IONavi , 2017, ACM Trans. Sens. Networks.

[6]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  D. Knuth Estimating the efficiency of backtrack programs. , 1974 .

[8]  Pheng-Ann Heng,et al.  A double-threshold image binarization method based on edge detector , 2008, Pattern Recognit..

[9]  Silvio Savarese,et al.  Semantic structure from motion , 2011, CVPR 2011.

[10]  Jan-Michael Frahm,et al.  Pixelwise View Selection for Unstructured Multi-View Stereo , 2016, ECCV.

[11]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Siyuan Liu,et al.  ShopProfiler: Profiling shops with crowdsourcing data , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[13]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Tamer Kahveci,et al.  RINQ: Reference-based Indexing for Network Queries , 2011, Bioinform..

[15]  Deke Guo,et al.  Enabling entity discovery in indoor commercial environments without pre-deployed infrastructure , 2017, Frontiers of Computer Science.

[16]  Justin Manweiler,et al.  OverLay: Practical Mobile Augmented Reality , 2015, MobiSys.

[17]  Tao Chen,et al.  From one to crowd: a survey on crowdsourcing-based wireless indoor localization , 2018, Frontiers of Computer Science.

[18]  Suman Nath,et al.  ALPS: accurate landmark positioning at city scales , 2016, UbiComp.

[19]  Philip S. Yu,et al.  Subgraph Matching with Set Similarity in a Large Graph Database , 2015, IEEE Transactions on Knowledge and Data Engineering.

[20]  Chunming Qiao,et al.  Rise of the Indoor Crowd: Reconstruction of Building Interior View via Mobile Crowdsourcing , 2015, SenSys.

[21]  Moustafa Youssef,et al.  CrowdInside: automatic construction of indoor floorplans , 2012, SIGSPATIAL/GIS.

[22]  Domenico Prattichizzo,et al.  Plane detection with stereo images , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[23]  Chunming Qiao,et al.  Crowd Map: Accurate Reconstruction of Indoor Floor Plans from Crowdsourced Sensor-Rich Videos , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[24]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[25]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[26]  Mo Li,et al.  Use it free: instantly knowing your phone attitude , 2014, MobiCom.

[27]  Frank Dürr,et al.  MapGENIE: Grammar-enhanced indoor map construction from crowd-sourced data , 2014, 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[28]  Kaigui Bian,et al.  Jigsaw: indoor floor plan reconstruction via mobile crowdsensing , 2014, MobiCom.

[29]  Robert P. Dick,et al.  Hallway based automatic indoor floorplan construction using room fingerprints , 2013, UbiComp.

[30]  Xi Fang,et al.  Crowdsourcing to smartphones: incentive mechanism design for mobile phone sensing , 2012, Mobicom '12.

[31]  Kaishun Wu,et al.  GRfid: A Device-Free RFID-Based Gesture Recognition System , 2017, IEEE Transactions on Mobile Computing.

[32]  Guobin Shen,et al.  Walkie-Markie: Indoor Pathway Mapping Made Easy , 2013, NSDI.

[33]  Silvio Savarese,et al.  Semantic structure from motion with points, regions, and objects , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Hirozumi Yamaguchi,et al.  TransitLabel: A Crowd-Sensing System for Automatic Labeling of Transit Stations Semantics , 2016, MobiSys.

[35]  Shijie Zhang,et al.  TreePi: A Novel Graph Indexing Method , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[36]  Mo Li,et al.  Travi-Navi: Self-Deployable Indoor Navigation System , 2017, TNET.

[37]  David F. Fouhey,et al.  Multiple Plane Detection in Image Pairs Using J-Linkage , 2010, 2010 20th International Conference on Pattern Recognition.

[38]  Srihari Nelakuditi,et al.  AutoLabel: labeling places from pictures and websites , 2016, UbiComp.

[39]  Mario Vento,et al.  A (sub)graph isomorphism algorithm for matching large graphs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Jiming Chen,et al.  Last-Mile Navigation Using Smartphones , 2015, MobiCom.

[41]  Ambuj K. Singh,et al.  Closure-Tree: An Index Structure for Graph Queries , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[42]  Kaishun Wu,et al.  WiFall: Device-free fall detection by wireless networks , 2017, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[43]  Rahul Sukthankar,et al.  Smarter Presentations: Exploiting Homography in Camera-Projector Systems , 2001, ICCV.

[44]  Silvio Savarese,et al.  Understanding the 3D layout of a cluttered room from multiple images , 2014, IEEE Winter Conference on Applications of Computer Vision.

[45]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[46]  Christine Solnon,et al.  A Comparison of Decomposition Methods for the Maximum Common Subgraph Problem , 2015, 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI).

[47]  Moustafa Youssef,et al.  SemanticSLAM: Using Environment Landmarks for Unsupervised Indoor Localization , 2016, IEEE Transactions on Mobile Computing.

[48]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[49]  Jignesh M. Patel,et al.  TALE: A Tool for Approximate Large Graph Matching , 2008, 2008 IEEE 24th International Conference on Data Engineering.