R3P: Real-time RGB-D Registration Pipeline

Applications based on colored 3-D data sequences suffer from lack of efficient algorithms for transformation estimation and key points extraction to perform accurate registration and sensor localization either in the 2-D or 3-D domain. Therefore, we propose a real-time RGB-D registration pipeline, named R3P, presented in processing layers. In this paper, we present an evaluation of several algorithm combinations for each layer, to optimize the registration and sensor localization for specific applications. The resulting dynamic reconfigurability of R3P makes it suitable as a front-end system for any SLAM reconstruction algorithm. Evaluation results on several public datasets reveal that R3P delivers real-time registration with 59 fps and high accuracy with the relative pose error (for a time span of 40 frames) for rotation and translation of \(0.5^\circ \) and 8 mm, respectively. All the heterogeneous dataset and implementations are publicly available under an open-source license [21].

[1]  Oscar Meruvia Pastor,et al.  DeReEs: Real-Time Registration of RGBD Images Using Image-Based Feature Detection And Robust 3D Correspondence Estimation and Refinement , 2014, IVCNZ '14.

[2]  François Charpillet,et al.  A real time visual SLAM for RGB-D cameras based on chamfer distance and occupancy grid , 2014, 2014 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[3]  Todor Stoyanov,et al.  Real time registration of RGB-D data using local visual features and 3D-NDT registration , 2012, ICRA 2012.

[4]  F. Michaud,et al.  Appearance-Based Loop Closure Detection for Online Large-Scale and Long-Term Operation , 2013, IEEE Transactions on Robotics.

[5]  Mark Pauly,et al.  Dynamic 2D/3D registration for the Kinect , 2013, SIGGRAPH '13.

[6]  Peter H. N. de With,et al.  Improved ICP-based pose estimation by Distance-aware 3D mapping , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[7]  Abdulmotaleb El-Saddik,et al.  A Combined Approach Toward Consistent Reconstructions of Indoor Spaces Based on 6D RGB-D Odometry and KinectFusion , 2015, ACM Trans. Intell. Syst. Technol..

[8]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[9]  Daniel Cremers,et al.  Real-time visual odometry from dense RGB-D images , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[10]  Henrik I. Christensen,et al.  RGB-D edge detection and edge-based registration , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Peter H. N. de With,et al.  Exploring Distance-Aware Weighting Strategies for Accurate Reconstruction of Voxel-Based 3D Synthetic Models , 2014, MMM.

[12]  Yan Lu,et al.  Robust RGB-D Odometry Using Point and Line Features , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  José García Rodríguez,et al.  A Comparative Study of Registration Methods for RGB-D Video of Static Scenes , 2014, Sensors.

[14]  Jörg Stückler,et al.  Model Learning and Real-Time Tracking Using Multi-Resolution Surfel Maps , 2012, AAAI.

[15]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[16]  Daniel Cremers,et al.  Robust odometry estimation for RGB-D cameras , 2013, 2013 IEEE International Conference on Robotics and Automation.

[17]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Wolfram Burgard,et al.  3-D Mapping With an RGB-D Camera , 2014, IEEE Transactions on Robotics.

[19]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[20]  Hani Javan Hemmat,et al.  Real-time planar segmentation of depth images , 2015 .

[21]  Gamini Dissanayake,et al.  A robust RGB-D SLAM algorithm , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Peter H. N. de With,et al.  Real-time planar segmentation of depth images: from three-dimensional edges to segmented planes , 2015, J. Electronic Imaging.

[23]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[24]  Andrew J. Davison,et al.  A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Dezhen Song,et al.  Visual Navigation Using Heterogeneous Landmarks and Unsupervised Geometric Constraints , 2015, IEEE Transactions on Robotics.

[26]  Alireza Bab-Hadiashar,et al.  A real-time RGB-D registration and mapping approach by heuristically switching between photometric and geometric information , 2014, 17th International Conference on Information Fusion (FUSION).

[27]  Andrew W. Fitzgibbon,et al.  Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.