BIM-PoseNet: Indoor camera localisation using a 3D indoor model and deep learning from synthetic images

Abstract The ubiquity of cameras built in mobile devices has resulted in a renewed interest in image-based localisation in indoor environments where the global navigation satellite system (GNSS) signals are not available. Existing approaches for indoor localisation using images either require an initial location or need first to perform a 3D reconstruction of the whole environment using structure-from-motion (SfM) methods, which is challenging and time-consuming for large indoor spaces. In this paper, a visual localisation approach is proposed to eliminate the requirement of image-based reconstruction of the indoor environment by using a 3D indoor model. A deep convolutional neural network (DCNN) is fine-tuned using synthetic images obtained from the 3D indoor model to regress the camera pose. Results of the experiments indicate that the proposed approach can be used for indoor localisation in real-time with an accuracy of approximately 2 m.

[1]  Valérie Gouet-Brunet,et al.  A survey on Visual-Based Localization: On the benefit of heterogeneous data , 2018, Pattern Recognit..

[2]  Peter I. Corke,et al.  Visual Place Recognition: A Survey , 2016, IEEE Transactions on Robotics.

[3]  Steffen Urban,et al.  SQUEEZEPOSENET: IMAGE BASED POSE REGRESSION WITH SMALL CONVOLUTIONAL NEURAL NETWORKS FOR REAL TIME UAS NAVIGATION , 2017 .

[4]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[5]  Ben Glocker,et al.  Real-Time RGB-D Camera Relocalization via Randomized Ferns for Keyframe Encoding , 2015, IEEE Transactions on Visualization and Computer Graphics.

[6]  Robert Harle,et al.  A Survey of Indoor Inertial Positioning Systems for Pedestrians , 2013, IEEE Communications Surveys & Tutorials.

[7]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[9]  George Vosselman,et al.  Mapping Indoor Spaces by Adaptive Coarse-to-Fine Registration of RGB-D Data , 2016, IEEE Geoscience and Remote Sensing Letters.

[10]  S. Winter,et al.  BIM-Tracker: A model-based visual tracking approach for indoor localisation using a 3D building model , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[11]  Cordelia Schmid,et al.  Learning from Synthetic Humans , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Wolfram Burgard,et al.  Deep Auxiliary Learning for Visual Localization and Odometry , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Kasun Hewage,et al.  Building Information Modeling (BIM) partnering framework for public construction projects , 2013 .

[14]  Chau Yuen,et al.  Indoor Positioning Using Visible LED Lights , 2015, ACM Comput. Surv..

[15]  Burcu Akinci,et al.  Analysis of Three Indoor Localization Technologies for Supporting Operations and Maintenance Field Tasks , 2012, J. Comput. Civ. Eng..

[16]  Hyoungkwan Kim,et al.  Image retrieval using BIM and features from pretrained VGG network for indoor localization , 2018, Building and Environment.

[17]  Ronald Raulefs,et al.  Recent Advances in Indoor Localization: A Survey on Theoretical Approaches and Applications , 2017, IEEE Communications Surveys & Tutorials.

[18]  Esa Rahtu,et al.  Relative Camera Pose Estimation Using Convolutional Neural Networks , 2017, ACIVS.

[19]  Paolo Valigi,et al.  Exploring Representation Learning With CNNs for Frame-to-Frame Ego-Motion Estimation , 2016, IEEE Robotics and Automation Letters.

[20]  Kourosh Khoshelham,et al.  Geometric comparison and quality evaluation of 3D models of indoor environments , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[21]  Agathoniki Trigoni,et al.  Lightweight map matching for indoor localisation using conditional random fields , 2014, IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks.

[22]  Kourosh Khoshelham,et al.  THE ISPRS BENCHMARK ON INDOOR MODELLING , 2017 .

[23]  Kourosh Khoshelham,et al.  AN EVALUATION FRAMEWORK FOR BENCHMARKING INDOOR MODELLING METHODS , 2018 .

[24]  Fernando Seco Granja,et al.  Improving indoor positioning using an efficient Map Matching and an extended motion model , 2015 .

[25]  Frank Schultmann,et al.  Building Information Modeling (BIM) for existing buildings — Literature review and future needs , 2014 .

[26]  Kourosh Khoshelham,et al.  INDOOR POSITIONING by VISUAL-INERTIAL ODOMETRY , 2017 .

[27]  Ondrej Chum,et al.  CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples , 2016, ECCV.

[28]  Fredrik Kahl,et al.  City-Scale Localization for Cameras with Known Vertical Direction , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.