Point Cloud Registration

In this chapter, we focus on keypoint-based point cloud registration which has proven to be among the most efficient strategies for aligning pairs of overlapping scans. We present a novel and fully automated framework which consists of six components addressing (i) the generation of 2D image representations in the form of range and intensity images, (ii) point quality assessment, (iii) feature extraction and matching, (iv) the forward projection of 2D keypoints to 3D space, (v) correspondence weighting, and (vi) point cloud registration. For the respective components, we take into account different approaches and our main contributions address the issue of how to increase the robustness, efficiency, and accuracy of point cloud registration by either introducing further constraints (e.g., addressing a correspondence weighting based on point quality measures) or replacing commonly applied approaches by more promising alternatives. The latter may not only address the involved strategy for point cloud registration, but also the involved approaches for feature extraction and matching. In a detailed evaluation, we demonstrate that, instead of directly aligning sets of corresponding 3D points, a transfer of the task of point cloud registration to the task of solving the Perspective-n-Point (PnP) problem or to the task of finding the relative orientation between sets of bearing vectors offers great potential for future research. Furthermore, our results clearly reveal that the further consideration of both a correspondence weighting based on point quality measures and a selection of an appropriate feature detector–descriptor combination may result in significant advantages with respect to robustness, efficiency, and accuracy.

[1]  Uwe Stilla,et al.  Simultaneous Calibration of ALS Systems and Alignment of Multiview LiDAR Scans of Urban Areas , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[2]  George Vosselman,et al.  An integrated approach for modelling and global registration of point clouds , 2007 .

[3]  Martin Weinmann,et al.  Visual Features - From Early Concepts to Modern Computer Vision , 2013, Advanced Topics in Computer Vision.

[4]  David Fofi,et al.  A review of recent range image registration methods with accuracy evaluation , 2007, Image Vis. Comput..

[5]  Jing Li,et al.  A comprehensive review of current local features for computer vision , 2008, Neurocomputing.

[6]  Wolfram Burgard,et al.  Robust place recognition for 3D range data based on point features , 2010, 2010 IEEE International Conference on Robotics and Automation.

[7]  Henrik Aanæs,et al.  Finding the Best Feature Detector-Descriptor Combination , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[8]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[9]  A. Gruen,et al.  Least squares 3D surface and curve matching , 2005 .

[10]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  S. Filin,et al.  Keypoint based autonomous registration of terrestrial laser point-clouds , 2008 .

[12]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[13]  Derek D. Lichti,et al.  A FRAMEWORK FOR POSITION UNCERTAINTY OF UNORGANISED THREE-DIMENSIONAL POINT CLOUDS FROM NEAR-MONOSTATIC LASER SCANNERS USING COVARIANCE ANALYSIS , 2005 .

[14]  Wolfram Burgard,et al.  Robust on-line model-based object detection from range images , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Uwe Stilla,et al.  Automatic Co-Registration of Airborne Laser-scanner Data Recorded at an Urban Area with Oblique Sensor Configuration , 2009 .

[16]  Andreas Birk,et al.  Fast Registration Based on Noisy Planes With Unknown Correspondences for 3-D Mapping , 2010, IEEE Transactions on Robotics.

[17]  S. Umeyama,et al.  Least-Squares Estimation of Transformation Parameters Between Two Point Patterns , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using orthonormal matrices , 1988 .

[20]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[21]  G. Vosselman,et al.  GENERATION AND WEIGHTING OF 3D POINT CORRESPONDENCES FOR IMPROVED REGISTRATION OF RGB-D DATA , 2013 .

[22]  Clive S. Fraser,et al.  Registration of terrestrial laser scanner data using imagery , 2006 .

[23]  Ioannis Stamos,et al.  Automated feature-based range registration of urban scenes of large scale , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[24]  Uwe Stilla,et al.  AUTOMATIC REGISTRATION OF LASER POINT CLOUDS OF URBAN AREAS , 2007 .

[25]  Stefan Hinz,et al.  Fast and automatic image-based registration of TLS data , 2011 .

[26]  Sang Wook Lee,et al.  Range data registration using photometric features , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Vincent Lepetit,et al.  Accurate Non-Iterative O(n) Solution to the PnP Problem , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[28]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Hanno Scharr,et al.  A Scheme for Coherence-Enhancing Diffusion Filtering with Optimized Rotation Invariance , 2002, J. Vis. Commun. Image Represent..

[30]  Sagi Filin,et al.  REGISTRATION OF TERRESTRIAL LASER SCANS VIA IMAGE BASED FEATURES , 2007 .

[31]  Nicolas David,et al.  Towards 3D lidar point cloud registration improvement using optimal neighborhood knowledge , 2013 .

[32]  Konrad Schindler,et al.  AUTOMATIC REGISTRATION OF TERRESTRIAL LASER SCANNER POINT CLOUDS USING NATURAL PLANAR SURFACES , 2012 .

[33]  Tom Drummond,et al.  Fusing points and lines for high performance tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[34]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[35]  Konrad Schindler,et al.  Keypoint-based 4-Points Congruent Sets – Automated marker-less registration of laser scans , 2014 .

[36]  B. Jutzi,et al.  Fast and accurate point cloud registration by exploiting inverse cumulative histograms (ICHs) , 2013, Joint Urban Remote Sensing Event 2013.

[37]  N. Mitra,et al.  4-points congruent sets for robust pairwise surface registration , 2008, SIGGRAPH 2008.

[38]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[39]  Vincent Lepetit,et al.  Learning Image Descriptors with the Boosting-Trick , 2012, NIPS.

[40]  David Nister,et al.  Recent developments on direct relative orientation , 2006 .

[41]  E. K. Forkuo.  Automatic fusion of photogrammetric imagery and laser scanner point clouds , 2003 .

[42]  Axel Wendt,et al.  A concept for feature based data registration by simultaneous consideration of laser scanner data and photogrammetric images , 2007 .

[43]  Roderik Lindenbergh,et al.  Optimizing terrestrial laser scanning measurement set-up , 2012 .

[44]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[45]  Kostas Daniilidis,et al.  Fully Automatic Registration of 3D Point Clouds , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[46]  Geraldine S. Cheok,et al.  Fast automatic registration of range images from 3D imaging systems using sphere targets , 2009 .

[47]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[48]  Zhanyi Hu,et al.  PnP Problem Revisited , 2005, Journal of Mathematical Imaging and Vision.

[49]  Peter Biber,et al.  The normal distributions transform: a new approach to laser scan matching , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[50]  Konrad Schindler,et al.  Fast registration of laser scans with 4-point congruent sets - what works and what doesn't , 2014 .

[51]  Olaf Hellwich,et al.  Comparison of 3D interest point detectors and descriptors for point cloud fusion , 2014, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[52]  Robert B. Fisher,et al.  Special Issue on Registration and Fusion of Range Images , 2002, Comput. Vis. Image Underst..

[53]  Fabio Remondino,et al.  AUTOMATIC REGISTRATION OF MULTIPLE LASER SCANS USING PANORAMIC RGB AND INTENSITY IMAGES , 2012 .

[54]  Derek D. Lichti,et al.  A method for automated registration of unorganised point clouds , 2008 .

[55]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[56]  Torsten Sattler,et al.  SCRAMSAC: Improving RANSAC's efficiency with a spatial consistency filter , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[57]  Jack Bresenham,et al.  Algorithm for computer control of a digital plotter , 1965, IBM Syst. J..

[58]  Steffen Urban,et al.  FINDING A GOOD FEATURE DETECTOR-DESCRIPTOR COMBINATION FOR THE 2D KEYPOINT-BASED REGISTRATION OF TLS POINT CLOUDS , 2015 .

[59]  Andrea Censi,et al.  An ICP variant using a point-to-line metric , 2008, 2008 IEEE International Conference on Robotics and Automation.

[60]  Jing Huang,et al.  Point cloud matching based on 3D self-similarity , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[61]  Martial Hebert,et al.  Fully automatic registration of multiple 3D data sets , 2003, Image Vis. Comput..

[62]  Boris Jutzi,et al.  FULLY AUTOMATIC IMAGE-BASED REGISTRATION OF UNORGANIZED TLS DATA , 2012 .

[63]  Robert B. Fisher,et al.  Estimating 3-D rigid body transformations: a comparison of four major algorithms , 1997, Machine Vision and Applications.

[64]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Stefan Hinz,et al.  AUTOMATIC FEATURE-BASED POINT CLOUD REGISTRATION FOR A MOVING SENSOR PLATFORM , 2013 .

[66]  Konrad Schindler,et al.  Markerless point cloud registration with keypoint-based 4-points congruent sets , 2013 .

[67]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[68]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[69]  Adrien Bartoli,et al.  Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces , 2013, BMVC.

[70]  Gary K. L. Tam,et al.  Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid , 2013, IEEE Transactions on Visualization and Computer Graphics.

[71]  Nico Blodow,et al.  Persistent Point Feature Histograms for 3D Point Clouds , 2008 .

[72]  Tom Duckett,et al.  Scan registration for autonomous mining vehicles using 3D‐NDT , 2007, J. Field Robotics.

[73]  Claus Brenner,et al.  Coarse orientation of terrestrial laser scans in urban environments , 2008 .

[74]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[75]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[76]  Berthold K. P. Horn.  Extended Gaussian images , 1984, Proceedings of the IEEE.

[77]  Ivan Detchev,et al.  Analysis of Two Triangle-Based Multi-Surface Registration Algorithms of Irregular Point Clouds , 2012 .

[78]  Laurent Kneip,et al.  OpenGV: A unified and generalized approach to real-time calibrated geometric vision , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[79]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[80]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[81]  Andreas Nüchter,et al.  Skyline-based registration of 3D laser scans , 2011, Geo spatial Inf. Sci..

[82]  Vincent Lepetit,et al.  Boosting Binary Keypoint Descriptors , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[83]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[84]  Sisi Zlatanova,et al.  Automatic Registration of Terrestrial Laser Scanning Point Clouds using Panoramic Reflectance Images , 2009, Sensors.

[85]  Vincent Lepetit,et al.  A fast local descriptor for dense matching , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[86]  Kari Pulli,et al.  Projective surface matching of colored 3D scans , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[87]  Boris Jutzi,et al.  A STEP TOWARDS DYNAMIC SCENE ANALYSIS WITH ACTIVE MULTI-VIEW RANGE IMAGING SYSTEMS , 2012 .

[88]  Marc Pollefeys,et al.  Automatic Registration of RGB-D Scans via Salient Directions , 2013, 2013 IEEE International Conference on Computer Vision.

[89]  Boris Jutzi,et al.  GEOMETRIC POINT QUALITY ASSESSMENT FOR THE AUTOMATED, MARKERLESS AND ROBUST REGISTRATION OF UNORDERED TLS POINT CLOUDS , 2015 .

[90]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[91]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[92]  Christoph Dold.  EXTENDED GAUSSIAN IMAGES FOR THE REGISTRATION OF TERRESTRIAL SCAN DATA , 2005 .

[93]  T. Lindeberg,et al.  Scale-Space Theory : A Basic Tool for Analysing Structures at Different Scales , 1994 .

[94]  Wolfram Burgard,et al.  Point feature extraction on 3D range scans taking into account object boundaries , 2011, 2011 IEEE International Conference on Robotics and Automation.

[95]  Susanne Becker,et al.  Automatic Marker-Free Registration of Terrestrial Laser Scans using Reflectance Features , 2007 .

[96]  Reinhard Klein,et al.  Image-Based Registration of 3D-Range Data Using Feature Surface Elements , 2004, VAST.

[97]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[98]  Florent Lamiraux,et al.  Metric-based iterative closest point scan matching for sensor displacement estimation , 2006, IEEE Transactions on Robotics.

[99]  Boris Jutzi,et al.  Semi-automatic image-based fusion of range imaging data with different characteristics , 2011 .

[100]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.