Motion segmentation based robust RGB-D SLAM

A sparse feature-based motion segmentation algorithm for RGB-D data is proposed which offers us a unified way to handle outliers and dynamic scenarios. Together with the pose-graph SLAM framework, they constitute an effective and robust solution that enable us to do RGB-D SLAM in wide range of situations, although traditionally they have been divided into different categories and treated separately using different kinds of methods. Through comparisons with RANSAC using simulated data and testing with different benchmark RGB-D datasets against the state-of-the-art method in RGB-D SLAM, we show that our solution is efficient and effective in handling general static and dynamic scenarios, some of which have not be achieved before.

[1]  Niko Sünderhauf Robust optimization for simultaneous localization and mapping , 2012 .

[2]  Ryan M. Eustice,et al.  Pose-graph visual SLAM with geometric model selection for autonomous underwater ship hull inspection , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Basilio Bona,et al.  A fast and accurate approximation for planar pose graph optimization , 2014, Int. J. Robotics Res..

[4]  Sebastian Thrun,et al.  FastSLAM: a factored solution to the simultaneous localization and mapping problem , 2002, AAAI/IAAI.

[5]  Yasuyuki Matsushita,et al.  Motion detail preserving optical flow estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Albert S. Huang,et al.  Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera , 2011, ISRR.

[7]  Kostas Daniilidis,et al.  Correspondence-free Structure from Motion , 2007, International Journal of Computer Vision.

[8]  W. Burgard,et al.  Real-time 3 D visual SLAM with a hand-held RGB-D camera , 2011 .

[9]  Wolfram Burgard,et al.  An evaluation of the RGB-D SLAM system , 2012, 2012 IEEE International Conference on Robotics and Automation.

[10]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[11]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[12]  Chieh-Chih Wang,et al.  Stereo-based simultaneous localization, mapping and moving object tracking , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[14]  John J. Leonard,et al.  Mapping the MIT Stata Center: Large-scale Integrated Visual and RGB-D SLAM , 2012 .

[15]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[16]  Heng Wang,et al.  On the number of local minima to the point feature based SLAM problem , 2012, 2012 IEEE International Conference on Robotics and Automation.

[17]  Gamini Dissanayake,et al.  Convergence and Consistency Analysis for Extended Kalman Filter Based SLAM , 2007, IEEE Transactions on Robotics.

[18]  Uwe Franke,et al.  6D-Vision: Fusion of Stereo and Motion for Robust Environment Perception , 2005, DAGM-Symposium.

[19]  Sunglok Choi,et al.  Performance Evaluation of RANSAC Family , 2009, BMVC.

[20]  É. Vincent,et al.  Detecting planar homographies in an image pair , 2001, ISPA 2001. Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis. In conjunction with 23rd International Conference on Information Technology Interfaces (IEEE Cat..

[21]  Gibson Hu,et al.  Towards reliability and scalability in feature based simultaneous localization and mapping , 2014 .

[22]  Yasir Latif,et al.  Robust graph SLAM back-ends: A comparative analysis , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[24]  Tat-Jun Chin,et al.  Simultaneously Fitting and Segmenting Multiple-Structure Data with Outliers , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Jitendra Malik,et al.  Object Segmentation by Long Term Analysis of Point Trajectories , 2010, ECCV.

[27]  Hugh F. Durrant-Whyte,et al.  Mobile robot localization by tracking geometric beacons , 1991, IEEE Trans. Robotics Autom..

[28]  Evangelos E. Milios,et al.  Globally Consistent Range Scan Alignment for Environment Mapping , 1997, Auton. Robots.

[29]  Hauke Strasdat,et al.  Visual SLAM: Why filter? , 2012, Image Vis. Comput..

[30]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[31]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[32]  Hauke Strasdat,et al.  Scale Drift-Aware Large Scale Monocular SLAM , 2010, Robotics: Science and Systems.

[33]  Udo Frese,et al.  Interview: Is SLAM Solved? , 2010, KI - Künstliche Intelligenz.

[34]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[35]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Lourdes Agapito,et al.  Dense Variational Reconstruction of Non-rigid Surfaces from Monocular Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  B. S. Manjunath,et al.  The multiRANSAC algorithm and its application to detect planar homographies , 2005, IEEE International Conference on Image Processing 2005.

[38]  Andreas Birk,et al.  Online three‐dimensional SLAM by registration of large planar surface segments and closed‐form pose‐graph relaxation , 2010, J. Field Robotics.

[39]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Niko Sünderhauf,et al.  Switchable constraints for robust pose graph SLAM , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  Yasir Latif,et al.  Robust Loop Closing Over Time , 2012, Robotics: Science and Systems.

[42]  John J. Leonard,et al.  Dynamic pose graph SLAM: Long-term mapping in low dynamic environments , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[43]  Stephen M. Smith,et al.  ASSET-2: Real-Time Motion Segmentation and Object Tracking , 1998, Real Time Imaging.

[44]  F. Fraundorfer,et al.  Visual Odometry : Part II: Matching, Robustness, Optimization, and Applications , 2012, IEEE Robotics & Automation Magazine.

[45]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[47]  Shoudong Huang,et al.  An efficient motion segmentation algorithm for multibody RGB-D slam , 2013 .

[48]  Vittorio Ferrari,et al.  Fast Object Segmentation in Unconstrained Video , 2013, 2013 IEEE International Conference on Computer Vision.

[49]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[50]  Hanzi Wang,et al.  AMSAC: An adaptive robust estimator for model fitting , 2013, 2013 IEEE International Conference on Image Processing.

[51]  Edwin Olson,et al.  Inference on networks of mixtures for robust robot mapping , 2012, Robotics: Science and Systems.

[52]  Konrad Schindler,et al.  Perspective n-View Multibody Structure-and-Motion Through Model Selection , 2006, ECCV.

[53]  Wolfram Burgard,et al.  G2o: A general framework for graph optimization , 2011, 2011 IEEE International Conference on Robotics and Automation.

[54]  Winston Churchill,et al.  Experience-based navigation for long-term localisation , 2013, Int. J. Robotics Res..

[55]  Nick Barnes,et al.  A Simple and Practical Solution to the Rigid Body Motion Segmentation Problem Using a RGB-D Camera , 2011, 2011 International Conference on Digital Image Computing: Techniques and Applications.

[56]  Kurt Konolige,et al.  Real-Time Detection of Independent Motion using Stereo , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[57]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[58]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[59]  Hugh F. Durrant-Whyte,et al.  Simultaneous Localization, Mapping and Moving Object Tracking , 2007, Int. J. Robotics Res..

[60]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[61]  K. Madhava Krishna,et al.  Realtime multibody visual SLAM with a smoothly moving monocular camera , 2011, 2011 International Conference on Computer Vision.

[62]  Daniel Cremers,et al.  Robust odometry estimation for RGB-D cameras , 2013, 2013 IEEE International Conference on Robotics and Automation.

[63]  Sebastian Thrun,et al.  FastSLAM 2.0: an improved particle filtering algorithm for simultaneous localization and mapping that provably converges , 2003, IJCAI 2003.

[64]  Tat-Jun Chin,et al.  Multi-structure model selection via kernel optimisation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[65]  Paolo Pirjanian,et al.  A Visual Front-end for Simultaneous Localization and Mapping , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[66]  Jitendra Malik,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Segmentation of Moving Objects by Long Term Video Analysis , 2022 .

[67]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[68]  Niko Sünderhauf,et al.  Towards a robust back-end for pose graph SLAM , 2012, 2012 IEEE International Conference on Robotics and Automation.

[69]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[70]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[71]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[72]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[73]  Eduardo Mario Nebot,et al.  Consistency of the EKF-SLAM Algorithm , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[74]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[75]  Lourdes Agapito,et al.  Dense multibody motion estimation and reconstruction from a handheld camera , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[76]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[77]  Wolfram Burgard,et al.  A Tutorial on Graph-Based SLAM , 2010, IEEE Intelligent Transportation Systems Magazine.

[78]  David Suter,et al.  A Model-Selection Framework for Multibody Structure-and-Motion of Image Sequences , 2007, International Journal of Computer Vision.

[79]  Pieter Abbeel,et al.  LQG-MP: Optimized path planning for robots with motion uncertainty and imperfect state information , 2010, Int. J. Robotics Res..

[80]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.