DeReEs: Real-Time Registration of RGBD Images Using Image-Based Feature Detection And Robust 3D Correspondence Estimation and Refinement

We present DeReEs, a real-time RGBD registration algorithm for the scenario where multiple RGBD images of the same scene are obtained from depth-sensing cameras placed at different viewpoints, with partial overlaps between their views. DeReEs (Detection, Rejection and Estimation) is a combination of 2D image-based feature detection algorithms, a RANSAC based false correspondence rejection and a rigid 3D transformation estimation. DeReEs performs global registration not only in real-time, but also supports large transformation distances for both translations and rotations. DeReEs is designed as part of a virtual/augmented reality solution for a remote 3D collaboration system that does not require initial setup and allows users to freely move the cameras during use. We present comparisons of DeReEs with other common registration algorithms. Our results suggest that DeReEs provides better speed and accuracy especially in scenes with partial overlapping.

[1]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[2]  Sebastian Weik,et al.  Registration of 3-D partial surface models using luminance and depth information , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[3]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[4]  Robert B. Fisher,et al.  Estimating 3-D rigid body transformations: a comparison of four major algorithms , 1997, Machine Vision and Applications.

[5]  Patrick Rives,et al.  An asymmetric real-time dense visual localisation and mapping system , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[6]  Ryan M. Eustice,et al.  Real-Time Visual SLAM for Autonomous Underwater Hull Inspection Using Visual Saliency , 2013, IEEE Transactions on Robotics.

[7]  Marc Pollefeys,et al.  Automatic Registration of RGB-D Scans via Salient Directions , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Christoph Stiller,et al.  Velodyne SLAM , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[9]  Peter Biber,et al.  The normal distributions transform: a new approach to laser scan matching , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[10]  Gang Wang,et al.  Registration and Integration of Multiple Object Views for 3D Model Construction , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Steven Mills,et al.  Reconstructing partially visible models using stereo vision, structured light, and the g2o framework , 2012, IVCNZ '12.

[13]  Michael G. Strintzis,et al.  A gesture recognition system using 3D data , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[14]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[15]  S. Druon,et al.  Color Constrained ICP for Registration of Large Unstructured 3D Color Data Sets , 2006, 2006 IEEE International Conference on Information Acquisition.

[16]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[18]  Ben Glocker,et al.  Real-time RGB-D camera relocalization , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[19]  Vladlen Koltun,et al.  Dense scene reconstruction with points of interest , 2013, ACM Trans. Graph..

[20]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[21]  Martin D. Levine,et al.  Registering Multiview Range Data to Create 3D Computer Objects , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Zoltan-Csaba Marton,et al.  On fast surface reconstruction methods for large and noisy point clouds , 2009, 2009 IEEE International Conference on Robotics and Automation.

[23]  Michael Beetz,et al.  Distinctive texture features from perspective-invariant keypoints , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[24]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[25]  Pushmeet Kohli,et al.  When Can We Use KinectFusion for Ground Truth Acquisition , 2012 .

[26]  Vincent Lepetit,et al.  Multiple 3D Object tracking for augmented reality , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[27]  Dieter Fox,et al.  RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments , 2010, ISER.

[28]  Jiri Matas,et al.  Robust Detection of Lines Using the Progressive Probabilistic Hough Transform , 2000, Comput. Vis. Image Underst..

[29]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  David Fofi,et al.  A comparative survey on invisible structured light , 2004, IS&T/SPIE Electronic Imaging.

[31]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[32]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[33]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[34]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[35]  Marc Rioux,et al.  Three-dimensional registration using range and intensity information , 1994, Other Conferences.

[36]  Paul Newman,et al.  Using laser range data for 3D SLAM in outdoor environments , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[37]  David Kim,et al.  Shake'n'sense: reducing interference for overlapping structured light depth cameras , 2012, CHI.

[38]  Tom Duckett,et al.  Scan registration for autonomous mining vehicles using 3D-NDT: Research Articles , 2007 .

[39]  Tamal K. Dey,et al.  Eurographics Symposium on Point-based Graphics (2005) Normal Estimation for Point Clouds: a Comparison Study for a Voronoi Based Method , 2022 .

[40]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[41]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Thomas G. Dietterich,et al.  Principal Curvature-Based Region Detector for Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Heinz Hügli,et al.  A multi-resolution ICP with heuristic closest point search for fast and robust 3D registration of range images , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[44]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[45]  Didier Stricker,et al.  Algorithms for 3D Shape Scanning with a Depth Camera , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Ruzena Bajcsy,et al.  Immersive 3D Environment for Remote Collaboration and Training of Physical Activities , 2008, 2008 IEEE Virtual Reality Conference.

[47]  Kesheng Wang,et al.  Accurate 3D object measurement and inspection using structured light systems , 2011, CompSysTech '11.

[48]  Patrick Rives,et al.  Real-time dense RGB-D localisation and mapping , 2011, IEEE International Conference on Robotics and Automation.

[49]  Jake K. Aggarwal,et al.  Human detection using depth information by Kinect , 2011, CVPR 2011 WORKSHOPS.

[50]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Tim Weyrich,et al.  A practical structured light acquisition system for point-based geometry and texture , 2005, Proceedings Eurographics/IEEE VGTC Symposium Point-Based Graphics, 2005..

[52]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[53]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[54]  Peter Johannes Neugebauer,et al.  Geometrical cloning of 3D objects via simultaneous registration of multiple range images , 1997, Proceedings of 1997 International Conference on Shape Modeling and Applications.

[55]  Katsuhiko Sakaue,et al.  Registration and integration of multiple range images for 3-D model construction , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[56]  Andreas Geiger,et al.  Visual SLAM for autonomous ground vehicles , 2011, 2011 IEEE International Conference on Robotics and Automation.

[57]  Andrew J. Hanson.  Visualizing Quaternions (The Morgan Kaufmann Series in Interactive 3D Technology) , 2006 .

[58]  Krystian Mikolajczyk,et al.  Evaluation of local detectors and descriptors for fast feature matching , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[59]  Martin Magnusson,et al.  The three-dimensional normal-distributions transform : an efficient representation for registration, surface analysis, and loop detection , 2009 .

[60]  John J. Leonard,et al.  Robust real-time visual odometry for dense RGB-D mapping , 2013, 2013 IEEE International Conference on Robotics and Automation.

[61]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  Ruzena Bajcsy,et al.  High-Quality Visualization for Geographically Distributed 3-D Teleimmersive Applications , 2011, IEEE Transactions on Multimedia.

[63]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[64]  Christof Lutteroth,et al.  Leveraging consumer sensing devices for telehealth , 2012, CHINZ '12.

[65]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[66]  Tom Duckett,et al.  Scan registration for autonomous mining vehicles using 3D‐NDT , 2007, J. Field Robotics.

[67]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[68]  Kari Pulli,et al.  Multiview registration for large data sets , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[69]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[70]  Andrew J. Davison,et al.  A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[71]  Dieter Fox,et al.  Interactive 3D modeling of indoor environments with a consumer depth camera , 2011, UbiComp '11.

[72]  Andrew Hogue,et al.  Underwater 3D SLAM through entropy minimization , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[73]  Manuela M. Veloso,et al.  Depth camera based indoor mobile robot localization and navigation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[74]  Roland Siegwart,et al.  Tracking a depth camera: Parameter exploration for fast ICP , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[75]  W. van der Mark,et al.  A comparative study of fast dense stereo vision algorithms , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[76]  Matthias Zwicker,et al.  Surface splatting , 2001, SIGGRAPH.

[77]  Daniel F. Huber,et al.  Automatic 3D modeling using range images obtained from unknown viewpoints , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.