RGB-D simultaneous localization and mapping based on combination of static point and line features in dynamic environments

Abstract. Visual simultaneous localization and mapping (SLAM) based on RGB-D data has been extensively researched in the past few years and has many robotic applications. Most of the state-of-the-art approaches assume static environments. However, the static assumption is not usually true in real world environments, dynamic objects can severely degrade the SLAM performance. In order to reduce the influence of dynamic objects on camera pose estimation, this paper proposes an approach that uses static point and line features. Static weights of point and line features indicating the likelihood of features being part of static environment are estimated. According to the calculated static weights, the data associated with dynamic objects are filtered out. The remaining static point and line features are considered inputs for refined pose estimation. Experiments are conducted with challenging dynamic sequences from TUM RGB-D dataset. The results demonstrate that the proposed approach is able to effectively improve the accuracy of RGB-D SLAM in dynamic environments.

[1]  Javier Gonzalez-Jimenez,et al.  PL-SLAM: A Stereo SLAM System Through the Combination of Points and Line Segments , 2017, IEEE Transactions on Robotics.

[2]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[4]  Mourad Zaied,et al.  Camera pose estimation for augmented reality in a small indoor dynamic scene , 2017, J. Electronic Imaging.

[5]  Yuxiang Sun,et al.  Improving RGB-D SLAM in dynamic environments: A motion removal approach , 2017, Robotics Auton. Syst..

[6]  Il Hong Suh,et al.  Building a 3-D Line-Based Map Using Stereo SLAM , 2015, IEEE Transactions on Robotics.

[7]  George Vosselman,et al.  Mapping Indoor Spaces by Adaptive Coarse-to-Fine Registration of RGB-D Data , 2016, IEEE Geoscience and Remote Sensing Letters.

[8]  Wolfram Burgard,et al.  3-D Mapping With an RGB-D Camera , 2014, IEEE Transactions on Robotics.

[9]  Hyun Myung,et al.  Solution to the SLAM Problem in Low Dynamic Environments Using a Pose Graph and an RGB-D Sensor , 2014, Sensors.

[10]  Yan Lu,et al.  Robust RGB-D Odometry Using Point and Line Features , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Daniel Cremers,et al.  Fast odometry and scene flow from RGB-D cameras based on geometric clustering , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Thomas Moore,et al.  A Generalized Extended Kalman Filter Implementation for the Robot Operating System , 2014, IAS.

[13]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[14]  Adrien Bartoli,et al.  The 3D Line Motion Matrix and Alignment of Line Reconstructions , 2004, International Journal of Computer Vision.

[15]  Andrew I. Comport,et al.  3D High Dynamic Range dense visual SLAM and its application to real-time object re-lighting , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[16]  Daniel Cremers,et al.  StaticFusion: Background Reconstruction for Dense RGB-D SLAM in Dynamic Environments , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Feng Zhu,et al.  Real-time depth camera tracking with geometrically stable weight algorithm , 2017 .

[18]  Dongheui Lee,et al.  RGB-D SLAM in Dynamic Environments Using Static Point Weighting , 2017, IEEE Robotics and Automation Letters.

[19]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[20]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[21]  Reinhard Koch,et al.  An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency , 2013, J. Vis. Commun. Image Represent..

[22]  Daniel Cremers,et al.  Robust odometry estimation for RGB-D cameras , 2013, 2013 IEEE International Conference on Robotics and Automation.

[23]  Jianhua Wang,et al.  Extension of an iterative closest point algorithm for simultaneous localization and mapping in corridor environments , 2016, J. Electronic Imaging.

[24]  Il Hong Suh,et al.  Building a partial 3D line-based map using a monocular SLAM , 2011, 2011 IEEE International Conference on Robotics and Automation.

[25]  Stefan Leutenegger,et al.  Dense RGB-D-inertial SLAM with map deformations , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Ian D. Reid,et al.  Real-Time Monocular SLAM with Straight Lines , 2006, BMVC.

[27]  Torsten Sattler,et al.  3D visual perception for self-driving cars using a multi-camera system: Calibration, mapping, localization, and obstacle detection , 2017, Image Vis. Comput..

[28]  Wolfram Burgard,et al.  An evaluation of the RGB-D SLAM system , 2012, 2012 IEEE International Conference on Robotics and Automation.

[29]  Jong-Hwan Kim,et al.  Effective Background Model-Based RGB-D Dense Visual Odometry in a Dynamic Environment , 2016, IEEE Transactions on Robotics.

[30]  Haibin Duan,et al.  Drogue pose estimation for unmanned aerial vehicle autonomous aerial refueling system based on infrared vision sensor , 2017 .

[31]  Michael Bosse,et al.  Keyframe-based visual–inertial odometry using nonlinear optimization , 2015, Int. J. Robotics Res..

[32]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Michal R. Nowicki,et al.  Accurate Map-Based RGB-D SLAM for Mobile Robots , 2015, ROBOT.