RGB-D SLAM in Dynamic Environments Using Points Correlations

In this paper, a simultaneous localization and mapping (SLAM) method that eliminates the influence of moving objects in dynamic environments is proposed. This method utilizes the correlation between map points to separate points that are part of the static scene and points that are part of different moving objects into different groups. A sparse graph is first created from all map points. In this graph, the vertices represent map points, and each edge represents the correlation between adjacent points. If the relative position between two points remains consistent over time, there is correlation between them, and they are considered to be moving together rigidly. If not, they are considered to have no correlation and to be in separate groups. After the edges between the uncorrelated points are removed, the remaining graph separates the map points of moving objects from the static scene. The largest group is assumed to be the group of reliable static map points. Experimental results demonstrate that robust and accurate performance can be achieved by the proposed SLAM method in both slightly and highly dynamic environments. Compared to other state-of-the-art methods, the proposed method can provide competitive accuracy with good real-time performance.

[1]  Wolfram Burgard,et al.  An evaluation of the RGB-D SLAM system , 2012, 2012 IEEE International Conference on Robotics and Automation.

[2]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[3]  Timothy D. Barfoot,et al.  State Estimation for Robotics , 2017 .

[4]  Truong Q. Nguyen,et al.  Visual odometry for RGB-D cameras for dynamic scenes , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  K. Madhava Krishna,et al.  Moving object detection by multi-view geometric techniques from a single camera mounted robot , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  U SaputraMuhamad Risqi,et al.  Visual SLAM and Structure from Motion in Dynamic Environments , 2018 .

[7]  Yoshihiko Nakamura,et al.  FlowFusion: Dynamic Dense RGB-D SLAM Based on Optical Flow , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Kurt Konolige,et al.  Incremental mapping of large cyclic environments , 1999, Proceedings 1999 IEEE International Symposium on Computational Intelligence in Robotics and Automation. CIRA'99 (Cat. No.99EX375).

[9]  Albert S. Huang,et al.  Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera , 2011, ISRR.

[10]  Daniel Cremers,et al.  StaticFusion: Background Reconstruction for Dense RGB-D SLAM in Dynamic Environments , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Jong-Hwan Kim,et al.  Effective Background Model-Based RGB-D Dense Visual Odometry in a Dynamic Environment , 2016, IEEE Transactions on Robotics.

[13]  Hujun Bao,et al.  Robust monocular SLAM in dynamic environments , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[14]  Michel Dhome,et al.  Real Time Localization and 3D Reconstruction , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[16]  Danping Zou,et al.  CoSLAM: Collaborative Visual SLAM in Dynamic Environments , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Jörg Stückler,et al.  Efficient Dense 3D Rigid-Body Motion Segmentation in RGB-D Video , 2013, BMVC.

[18]  Shaojie Shen,et al.  Tracking 3-D Motion of Dynamic Objects Using Monocular Visual-Inertial Sensing , 2019, IEEE Transactions on Robotics.

[19]  Daniel Cremers,et al.  Real-Time Dense Geometry from a Handheld Camera , 2010, DAGM-Symposium.

[20]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[21]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[22]  Michael Gassner,et al.  SVO: Semidirect Visual Odometry for Monocular and Multicamera Systems , 2017, IEEE Transactions on Robotics.

[23]  F. Fraundorfer,et al.  Visual Odometry : Part II: Matching, Robustness, Optimization, and Applications , 2012, IEEE Robotics & Automation Magazine.

[24]  José Ruíz Ascencio,et al.  Visual simultaneous localization and mapping: a survey , 2012, Artificial Intelligence Review.

[25]  Pavel Zemcík,et al.  Incremental Block Cholesky Factorization for Nonlinear Least Squares in Robotics , 2013, Robotics: Science and Systems.

[26]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[28]  H. Jin Kim,et al.  Robust Real-time RGB-D Visual Odometry in Dynamic Environments via Rigid Motion Model , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Lourdes Agapito,et al.  MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects , 2018, 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[30]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[31]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[32]  Yuxiang Sun,et al.  Improving RGB-D SLAM in dynamic environments: A motion removal approach , 2017, Robotics Auton. Syst..

[33]  Jizhong Xiao,et al.  Fast visual odometry and mapping from RGB-D data , 2013, 2013 IEEE International Conference on Robotics and Automation.

[34]  Wolfram Burgard,et al.  3-D Mapping With an RGB-D Camera , 2014, IEEE Transactions on Robotics.

[35]  Daniel Cremers,et al.  Robust odometry estimation for RGB-D cameras , 2013, 2013 IEEE International Conference on Robotics and Automation.

[36]  Jörg Stückler,et al.  EM-Fusion: Dynamic Object-Level SLAM With Probabilistic Data Association , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Dongheui Lee,et al.  RGB-D SLAM in Dynamic Environments Using Static Point Weighting , 2017, IEEE Robotics and Automation Letters.

[38]  Davide Scaramuzza,et al.  REMODE: Probabilistic, monocular dense reconstruction in real time , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Shichao Yang,et al.  CubeSLAM: Monocular 3-D Object SLAM , 2018, IEEE Transactions on Robotics.

[40]  Tim D. Barfoot,et al.  At all Costs: A Comparison of Robust Cost Functions for Camera Correspondence Outliers , 2015, 2015 12th Conference on Computer and Robot Vision.

[41]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[42]  Marc Pollefeys,et al.  Robust Dense Mapping for Large-Scale Dynamic Environments , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[43]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Zheng Fang,et al.  Experimental Evaluation of RGB-D Visual Odometry Methods , 2015 .

[45]  Jian Sun,et al.  Instance-Aware Semantic Segmentation via Multi-task Network Cascades , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Christoph Stiller,et al.  Moving on to dynamic environments: Visual odometry using feature classification , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  Javier Civera,et al.  DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes , 2018, IEEE Robotics and Automation Letters.

[48]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[49]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[50]  Daniel Cremers,et al.  Fast odometry and scene flow from RGB-D cameras based on geometric clustering , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[51]  Luis Miguel Bergasa,et al.  On combining visual SLAM and dense scene flow to increase the robustness of localization and mapping in dynamic environments , 2012, 2012 IEEE International Conference on Robotics and Automation.

[52]  Cyrill Stachniss,et al.  ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[53]  Cyrill Stachniss,et al.  Simultaneous Localization and Mapping , 2016, Springer Handbook of Robotics, 2nd Ed..

[54]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[55]  Kurt Konolige,et al.  g 2 o: A general Framework for (Hyper) Graph Optimization , 2011 .

[56]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[57]  Jong-Hwan Kim,et al.  Visual Odometry Algorithm Using an RGB-D Sensor and IMU in a Highly Dynamic Environment , 2014, RiTA.

[58]  Binbin Xu,et al.  MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[59]  Daniel Cremers,et al.  Direct Sparse Odometry , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Luis Montano,et al.  Semantic visual SLAM in populated environments , 2017, 2017 European Conference on Mobile Robots (ECMR).

[61]  Stefano Soatto,et al.  Structure from Motion Causally Integrated Over Time , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Ali Farhadi,et al.  Re$^3$: Re al-Time Recurrent Regression Networks for Visual Tracking of Generic Objects , 2017, IEEE Robotics and Automation Letters.

[63]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[64]  Matteo Matteucci,et al.  Use a Single Camera for Simultaneous Localization And Mapping with Mobile Object Tracking in dynamic environments , 2009, ICRA 2009.

[65]  Sebastian Thrun,et al.  Simultaneous Localization and Mapping , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[66]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[67]  Daniel Cremers,et al.  Real-time visual odometry from dense RGB-D images , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[68]  Hauke Strasdat,et al.  Visual SLAM: Why filter? , 2012, Image Vis. Comput..

[69]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[70]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[71]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[72]  Lourdes Agapito,et al.  Co-fusion: Real-time segmentation, tracking and fusion of multiple objects , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[73]  Tai-Jiang Mu,et al.  ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).