Registration of large-scale terrestrial laser scanner point clouds: A review and benchmark

Abstract This study had two main aims: (1) to provide a comprehensive review of terrestrial laser scanner (TLS) point cloud registration methods and a better understanding of their strengths and weaknesses; and (2) to provide a large-scale benchmark data set (Wuhan University TLS: Whu-TLS) to support the development of cutting-edge TLS point cloud registration methods, especially deep learning-based methods. In particular, we first conducted a thorough review of TLS point cloud registration methods in terms of pairwise coarse registration, pairwise fine registration, and multiview registration, as well as analyzing their strengths, weaknesses, and future research trends. We then reviewed the existing benchmark data sets (e.g., ETH Dataset and Robotic 3D Scanning Repository) for TLS point cloud registration and summarized their limitations. Finally, a new benchmark data set was assembled from 11 different environments (i.e., subway station, high-speed railway platform, mountain, forest, park, campus, residence, riverbank, heritage building, underground excavation, and tunnel environments) with variations in the point density, clutter, and occlusion. In addition, we summarized future research trends in this area, including auxiliary data-guided registration, deep learning-based registration, and multi-temporal point cloud registration.

[1]  Uwe Stilla,et al.  Learning a constrained conditional random field for enhanced segmentation of fallen trees in ALS point clouds , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[2]  Bisheng Yang,et al.  Hierarchical registration of unordered TLS point clouds based on binary shape context descriptor , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[3]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Junhao Xiao,et al.  Three-dimensional point cloud plane segmentation in both structured and unstructured environments , 2013, Robotics Auton. Syst..

[5]  Xuming Ge,et al.  Non-rigid registration of 3D point clouds under isometric deformation , 2016 .

[6]  Konrad Schindler,et al.  Fast registration of laser scans with 4-point congruent sets - what works and what doesn't , 2014 .

[7]  Sven Oesau,et al.  Indoor scene reconstruction using feature sensitive primitive extraction and graph-cut , 2014 .

[8]  Mohammed Bennamoun,et al.  An Accurate and Robust Range Image Registration Algorithm for 3D Object Modeling , 2014, IEEE Transactions on Multimedia.

[9]  Slobodan Ilic,et al.  PPFNet: Global Context Aware Local Features for Robust 3D Point Matching , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Bisheng Yang,et al.  A novel binary shape context for 3D local surface description , 2017 .

[11]  Tom Duckett,et al.  Scan registration for autonomous mining vehicles using 3D‐NDT , 2007, J. Field Robotics.

[12]  Javier Ruiz Hidalgo,et al.  Correspondence matching in unorganized 3D point clouds using Convolutional Neural Networks , 2019, Image Vis. Comput..

[13]  Venu Madhav Govindu,et al.  On Averaging Multiview Relations for 3D Scan Registration , 2014, IEEE Transactions on Image Processing.

[14]  Mingwei Wang,et al.  Entropy-Based Registration of Point Clouds Using Terrestrial Laser Scanning and Smartphone GPS , 2017, Sensors.

[15]  Olaf Hellwich,et al.  Automatic registration of unordered point clouds acquired by Kinect sensors using an overlap heuristic , 2015 .

[16]  Carlo Atzeni,et al.  Integration of Radar Interferometry and Laser Scanning for Remote Monitoring of an Urban Site Built on a Sliding Slope , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Jiaolong Yang,et al.  Go-ICP: Solving 3D Registration Efficiently and Globally Optimally , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[19]  Xuming Ge,et al.  Automatic markerless registration of point clouds with semantic-keypoint-based 4-points congruent sets , 2017 .

[20]  Bisheng Yang,et al.  Automatic registration of large-scale urban scene point clouds based on semantic feature points , 2016 .

[21]  Gary K. L. Tam,et al.  Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid , 2013, IEEE Transactions on Visualization and Computer Graphics.

[22]  Radu Horaud,et al.  Joint Alignment of Multiple Point Sets with Batch and Incremental Expectation-Maximization , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Lin Wang,et al.  Local to global registration of multi-view range scans using spanning tree , 2017, Comput. Electr. Eng..

[24]  Mohammed Bennamoun,et al.  A novel local surface feature for 3D object recognition under clutter and occlusion , 2015, Inf. Sci..

[25]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Andreas Nchter.  3D Robotic Mapping: The Simultaneous Localization and Mapping Problem with Six Degrees of Freedom , 2009 .

[27]  Yusheng Xu,et al.  Pairwise coarse registration of point clouds in urban scenes using voxel-based 4-planes congruent sets , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[28]  Jida Huang,et al.  V4PCS: Volumetric 4PCS Algorithm for Global Registration , 2017 .

[29]  Chen Li,et al.  Automatic multi-view registration of unordered range scans without feature extraction , 2016, Neurocomputing.

[30]  Alexander Prokop,et al.  Assessing the capability of terrestrial laser scanning for monitoring slow moving landslides , 2009 .

[31]  Martial Hebert,et al.  Fully automatic registration of multiple 3D data sets , 2003, Image Vis. Comput..

[32]  Bo Wu,et al.  Image-Guided Registration of Unordered Terrestrial Laser Scanning Point Clouds for Urban Scenes , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Xuming Ge,et al.  Surface-based matching of 3D point clouds with variable coordinates in source and target system , 2016 .

[34]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Uwe Stilla,et al.  A FRAMEWORK FOR VOXEL-BASED GLOBAL SCALE MODELING OF URBAN ENVIRONMENTS , 2016 .

[36]  Michael Greenspan,et al.  Generalized 4-Points Congruent Sets for 3D Registration , 2014, 2014 2nd International Conference on 3D Vision.

[37]  Dong Tian,et al.  FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  Max Q.-H. Meng,et al.  Robust Generalized Point Set Registration using Inhomogeneous Hybrid Mixture Models via Expectation Maximization , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[39]  Ping Wang,et al.  A refined coherent point drift (CPD) algorithm for point set registration , 2011, Science China Information Sciences.

[40]  Steven Lake Waslander,et al.  Scan registration using segmented region growing NDT , 2014, Int. J. Robotics Res..

[41]  Juha Hyyppä,et al.  Automatic Stem Mapping Using Single-Scan Terrestrial Laser Scanning , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[42]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[43]  Michael Greenspan,et al.  Super Generalized 4PCS for 3D Registration , 2015, 2015 International Conference on 3D Vision.

[44]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Federico Tombari,et al.  Performance Evaluation of 3D Keypoint Detectors , 2012, International Journal of Computer Vision.

[46]  Takashi Tsubouchi,et al.  A 3-D Scan Matching using Improved 3-D Normal Distributions Transform for Mobile Robotic Mapping , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  Uwe Stilla,et al.  Simultaneous Calibration of ALS Systems and Alignment of Multiview LiDAR Scans of Urban Areas , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[48]  George Vosselman,et al.  An integrated approach for modelling and global registration of point clouds , 2007 .

[49]  Niloy J. Mitra,et al.  Super4PCS: Fast Global Pointcloud Registration via Smart Indexing , 2019 .

[50]  Mohammed Bennamoun,et al.  Rotational Projection Statistics for 3D Local Surface Description and Object Recognition , 2013, International Journal of Computer Vision.

[51]  Higinio González-Jorge,et al.  4-Plane congruent sets for automatic registration of as-is 3D point clouds with 3D BIM models , 2018 .

[52]  Martin Weinmann,et al.  Book Review–Reconstruction and Analysis of 3D Scenes: From Irregularly Distributed 3D Points to Object Classes , 2016, Photogrammetric Engineering & Remote Sensing.

[53]  Baba C. Vemuri,et al.  Robust Point Set Registration Using Gaussian Mixture Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  George Vosselman,et al.  Airborne and terrestrial laser scanning , 2011, Int. J. Digit. Earth.

[55]  Mohammed Bennamoun,et al.  Three-Dimensional Model-Based Object Recognition and Segmentation in Cluttered Scenes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Nicolas David,et al.  Towards 3D lidar point cloud registration improvement using optimal neighborhood knowledge , 2013 .

[58]  Konrad Schindler,et al.  Keypoint-based 4-Points Congruent Sets – Automated marker-less registration of laser scans , 2014 .

[59]  Claus Brenner,et al.  Registration of terrestrial laser scanning data using planar patches and image data , 2006 .

[60]  Jonathan Li,et al.  Pairwise registration of TLS point clouds using covariance descriptors and a non-cooperative game , 2017 .

[61]  Cheng Wang,et al.  3-D Deep Feature Construction for Mobile Laser Scanning Point Cloud Registration , 2019, IEEE Geoscience and Remote Sensing Letters.

[62]  Jihua Zhu,et al.  Efficient registration of multi-view point sets by K-means clustering , 2019, Inf. Sci..

[63]  Uwe Stilla,et al.  AUTOMATED COARSE REGISTRATION OF POINT CLOUDS IN 3D URBAN SCENESUSING VOXEL BASED PLANE CONSTRAINT , 2017 .

[64]  Patrick J. Flynn,et al.  A Survey Of Free-Form Object Representation and Recognition Techniques , 2001, Comput. Vis. Image Underst..

[65]  Bisheng Yang,et al.  Automated registration of dense terrestrial laser-scanning point clouds using curves , 2014 .

[66]  Jieqing Feng,et al.  Hierarchical Multiview Rigid Registration , 2015, SGP '15.

[67]  Hao Xu,et al.  Registration of Laser Scanning Point Clouds: A Review , 2018, Sensors.

[68]  Sang Wook Lee,et al.  ICP Registration Using Invariant Features , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[69]  Jaehoon Jung,et al.  Productive modeling for development of as-built BIM of existing indoor structures , 2014 .

[70]  Derek D. Lichti,et al.  A method for automated registration of unorganised point clouds , 2008 .

[71]  David Fofi,et al.  A review of recent range image registration methods with accuracy evaluation , 2007, Image Vis. Comput..

[72]  Sheng-Wen Shih,et al.  An Efficient and Accurate Method for the Relaxation of Multiview Registration Error , 2008, IEEE Transactions on Image Processing.

[73]  Daniel P. Huttenlocher,et al.  Fast affine point matching: an output-sensitive method , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[74]  Yaxin Peng,et al.  LieTrICP: An improvement of trimmed iterative closest point algorithm , 2014, Neurocomputing.

[75]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  T Kanade,et al.  Techniques for fast and accurate intrasurgical registration. , 1995, Journal of image guided surgery.

[77]  M. Brito,et al.  Solar energy potential on roofs and facades in an urban landscape , 2013 .

[78]  P. Zingaretti,et al.  Performance evaluation of automated approaches to building detection in multi-source aerial data , 2010 .

[79]  Achim J. Lilienthal,et al.  Fast and accurate scan registration through minimization of the distance between compact 3D NDT representations , 2012, Int. J. Robotics Res..

[80]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Uwe Stilla,et al.  Acquisition and Consecutive Registration of Photogrammetric Point Clouds for Construction Progress Monitoring Using a 4D BIM , 2017, PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science.

[82]  Miguel Brito,et al.  Modelling solar potential in the urban environment: State-of-the-art review , 2015 .

[83]  Jan van Aardt,et al.  Marker-Free Registration of Forest Terrestrial Laser Scanner Data Pairs With Embedded Confidence Metrics , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[84]  Roderik Lindenbergh,et al.  AN IMPROVED COHERENT POINT DRIFT METHOD FOR TLS POINT CLOUD REGISTRATION OF COMPLEX SCENES , 2019, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[85]  Han Hu,et al.  Object-based incremental registration of terrestrial point clouds in an urban environment , 2020 .

[86]  Andrew W. Fitzgibbon.  Robust registration of 2D and 3D point sets , 2003, Image Vis. Comput..

[87]  Tat-Jun Chin,et al.  Practical optimal registration of terrestrial LiDAR scan pairs , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[88]  Uwe Stilla,et al.  FAST PAIRWISE COARSE REGISTRATION BETWEEN POINT CLOUDS OF CONSTRUCTION SITES USING 2D PROJECTION BASED PHASE CORRELATION , 2019 .

[89]  Bisheng Yang,et al.  An efficient global energy optimization approach for robust 3D plane segmentation of point clouds , 2018 .

[90]  Steven Lake Waslander,et al.  Scan registration with multi-scale k-means normal distributions transform , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[91]  Mohammed Bennamoun,et al.  A Comprehensive Performance Evaluation of 3D Local Feature Descriptors , 2015, International Journal of Computer Vision.

[92]  J. D. Wegner,et al.  Globally consistent registration of terrestrial laser scans via graph optimization , 2015 .