Drift-Aware Monocular Localization Based on a Pre-Constructed Dense 3D Map in Indoor Environments

Recently, monocular localization has attracted increased attention due to its application to indoor navigation and augmented reality. In this paper, a drift-aware monocular localization system that performs global and local localization is presented based on a pre-constructed dense three-dimensional (3D) map. In global localization, a pixel-distance weighted least squares algorithm is investigated for calculating the absolute scale for the epipolar constraint. To reduce the accumulative errors that are caused by the relative position estimation, a map interaction-based drift detection method is introduced in local localization, and the drift distance is computed by the proposed line model-based maximum likelihood estimation sample consensus (MLESAC) algorithm. The line model contains a fitted line segment and some visual feature points, which are used to seek inliers of the estimated feature points for drift detection. Taking advantage of the drift detection method, the monocular localization system switches between the global and local localization modes, which effectively keeps the position errors within an expected range. The performance of the proposed monocular localization system is evaluated on typical indoor scenes, and experimental results show that compared with the existing localization methods, the accuracy improvement rates of the absolute position estimation and the relative position estimation are at least 30.09% and 65.59%, respectively.

[1]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[2]  Judith Dijk,et al.  Closed Form Solution for the Scale Ambiguity Problem in Monocular Visual Odometry , 2010, ICIRA.

[3]  Kenichi Kanatani,et al.  Triangulation from Two Views Revisited: Hartley-Sturm vs. Optimal Correction , 2008, BMVC.

[4]  Alexandru Tupan,et al.  Triangulation , 1997, Comput. Vis. Image Underst..

[5]  Chung-Hao Huang,et al.  Real-Time RFID Indoor Positioning System Based on Kalman-Filter Drift Removal and Heron-Bilateration Location Estimation , 2015, IEEE Transactions on Instrumentation and Measurement.

[6]  Antonio Moschitta,et al.  Positioning Techniques in Indoor Environments Based on Stochastic Modeling of UWB Round-Trip-Time Measurements , 2016, IEEE Transactions on Intelligent Transportation Systems.

[7]  Florian Schweiger,et al.  TUMindoor: An extensive image and point cloud dataset for visual indoor localization and mapping , 2012, 2012 19th IEEE International Conference on Image Processing.

[8]  Edith Deretey.  Visual Localization in Underground Mines and Indoor Environments using PnP , 2016 .

[9]  Lina María Paz,et al.  Large-Scale 6-DOF SLAM With Stereo-in-Hand , 2008, IEEE Transactions on Robotics.

[10]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[11]  Michael Greenspan,et al.  Visual indoor positioning with a single camera using PnP , 2015, 2015 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[12]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[13]  Tilo Strutz,et al.  Data Fitting and Uncertainty: A practical introduction to weighted least squares and beyond , 2010 .

[14]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[16]  Shahrokh Valaee,et al.  2DTriPnP: A Robust Two-Dimensional Method for Fine Visual Localization Using Google Streetview Database , 2017, IEEE Transactions on Vehicular Technology.

[17]  Changzhi Wang,et al.  Intelligent RFID Indoor Localization System Using a Gaussian Filtering Based Extreme Learning Machine , 2017, Symmetry.

[18]  Huai-Rong Shao,et al.  WiFi-based indoor positioning , 2015, IEEE Communications Magazine.

[19]  Kun Yang,et al.  A Map-Assisted WiFi AP Placement Algorithm Enabling Mobile Device’s Indoor Positioning , 2017, IEEE Systems Journal.

[20]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Lin Ma,et al.  Visual Map Construction Using RGB-D Sensors for Image-Based Localization in Indoor Environments , 2017, J. Sensors.

[22]  Qiang Ji,et al.  Breakpoint Detection Using Covariance Propagation , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Harald Schuh,et al.  Precise positioning with current multi-constellation Global Navigation Satellite Systems: GPS, GLONASS, Galileo and BeiDou , 2015, Scientific Reports.

[24]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Michel Dhome,et al.  Real Time Localization and 3D Reconstruction , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Mohinder S. Grewal,et al.  Global navigation satellite systems , 2011 .

[27]  Shahrokh Valaee,et al.  A weighted KNN epipolar geometry-based approach for vision-based indoor localization using smartphone cameras , 2014, 2014 IEEE 8th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[28]  Gianluigi Ferrari,et al.  UWB-based localization in large indoor scenarios: optimized placement of anchor nodes , 2015, IEEE Transactions on Aerospace and Electronic Systems.

[29]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[30]  金谷 健一.  Statistical optimization for geometric computation : theory and practice , 2005 .

[31]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[32]  Lin Ma,et al.  Smart phone camera image localization method for narrow corridors based on epipolar geometry , 2016, 2016 International Wireless Communications and Mobile Computing Conference (IWCMC).

[33]  Roland Siegwart,et al.  Appearance-Guided Monocular Omnidirectional Visual Odometry for Outdoor Ground Vehicles , 2008, IEEE Transactions on Robotics.

[34]  Wolfram Burgard,et al.  G2o: A general framework for graph optimization , 2011, 2011 IEEE International Conference on Robotics and Automation.

[35]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[36]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[37]  David G. Lowe,et al.  Local and global localization for mobile robots using visual landmarks , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[38]  Daniel Herrera C,et al.  Joint depth and color camera calibration with distortion correction. , 2012, IEEE transactions on pattern analysis and machine intelligence.

[39]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[40]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[42]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[43]  Jaehyun Park,et al.  Resolving scale ambiguity for monocular visual odometry , 2013, 2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[44]  Ji Hoon Joung,et al.  What does ground tell us? Monocular visual odometry under planar motion constraint , 2011, 2011 11th International Conference on Control, Automation and Systems.