Self-Supervised Point Cloud Registration With Deep Versatile Descriptors for Intelligent Driving

As a fundamental yet challenging problem in intelligent transportation systems, point cloud registration attracts vast attention and has been attained with various deep learning-based algorithms. The unsupervised registration algorithms take advantage of deep neural network-enabled novel representation learning while requiring no human annotations, making them applicable to industrial applications. However, unsupervised methods mainly depend on global descriptors, which ignore the high-level representations of local geometries. In this paper, we propose to jointly use both global and local descriptors to register point clouds in a self-supervised manner, which is motivated by a critical observation that all local geometries of point clouds are transformed consistently under the same transformation. Therefore, local geometries can be employed to enhance the representation ability of the feature extraction module. Moreover, the proposed local descriptor is flexible and can be integrated into most existing registration methods and improve their performance. Besides, we also utilize point cloud reconstruction and normal estimation to enhance the transformation awareness of global and local descriptors. Lastly, extensive experimental results on one synthetic and three real-world datasets demonstrate that our method outperforms existing state-of-art unsupervised registration methods and even surpasses supervised ones in some cases. Robustness and computational efficiency evaluations also indicate that the proposed method applies to intelligent vehicles.

[1]  Jiwen Lu,et al.  PointGLR: Unsupervised Structural Representation Learning of 3D Point Clouds , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jie Tang,et al.  Self-Supervised Learning: Generative or Contrastive , 2020, IEEE Transactions on Knowledge and Data Engineering.

[3]  T. Truong,et al.  SAKS: Sampling Adaptive Kernels From Subspace for Point Cloud Graph Convolution , 2023, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Yujie Li,et al.  Global-PBNet: A Novel Point Cloud Registration for Autonomous Driving , 2022, IEEE Transactions on Intelligent Transportation Systems.

[5]  Ming Yang,et al.  Point Cloud Registration Based on Direct Deep Features With Applications in Intelligent Vehicles , 2022, IEEE Transactions on Intelligent Transportation Systems.

[6]  Dongrui Liu,et al.  PFMixer: Point Cloud Frequency Mixing , 2022, IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium.

[7]  Yuxing Peng,et al.  Geometric Transformer for Fast and Robust Point Cloud Registration , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Han Jiang,et al.  Robust Lane Extraction From MLS Point Clouds Towards HD Maps Especially in Curve Road , 2022, IEEE Transactions on Intelligent Transportation Systems.

[9]  Robert Caiming Qiu,et al.  A Robust and Reliable Point Cloud Recognition Network Under Rigid Transformation , 2020, IEEE Transactions on Instrumentation and Measurement.

[10]  Ziyong Feng,et al.  Point Cloud Registration using Representative Overlapping Points , 2021, ArXiv.

[11]  Simon Lucey,et al.  PointNetLK Revisited , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Kjell G. Robbersmyr,et al.  Toward Self-Supervised Feature Learning for Online Diagnosis of Multiple Faults in Electric Powertrains , 2021, IEEE Transactions on Industrial Informatics.

[13]  Xiaoshui Huang,et al.  A comprehensive survey on point cloud registration , 2021, ArXiv.

[14]  Bing Zeng,et al.  OMNet: Learning Overlapping Mask for Partial-to-Partial Point Cloud Registration , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[15]  Li-Ta Hsu,et al.  GNSS NLOS Exclusion Based on Dynamic Object Detection Using LiDAR Point Cloud , 2021, IEEE Transactions on Intelligent Transportation Systems.

[16]  Bo Yang,et al.  SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Xinlei Chen,et al.  Exploring Simple Siamese Representation Learning , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Gal Chechik,et al.  Self-Supervised Learning for Domain Adaptation on Point Clouds , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[19]  Heng Yang,et al.  TEASER: Fast and Certifiable Point Cloud Registration , 2020, IEEE Transactions on Robotics.

[20]  Yingli Tian,et al.  Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Juntong Xi,et al.  A Comprehensive Performance Evaluation of 3-D Transformation Estimation Techniques in Point Cloud Registration , 2021, IEEE Transactions on Instrumentation and Measurement.

[22]  Yi An,et al.  Lightweight Attention Module for Deep Learning on Classification and Segmentation of 3-D Point Clouds , 2021, IEEE Transactions on Instrumentation and Measurement.

[23]  Jin Wu,et al.  Rigid 3-D Registration: A Simple Method Free of SVD and Eigendecomposition , 2020, IEEE Transactions on Instrumentation and Measurement.

[24]  Jan Kautz,et al.  DeepGMR: Learning Latent Gaussian Mixture Models for Registration , 2020, ECCV.

[25]  Jian Zhang,et al.  Feature-Metric Registration: A Fast Semi-Supervised Approach for Robust Point Cloud Registration Without Correspondences , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Vladlen Koltun,et al.  Deep Global Registration , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Zi Jian Yew,et al.  RPM-Net: Robust Point Matching Using Learned Features , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Jie Zhou,et al.  Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Bin Liang,et al.  Virtual Stereovision Pose Measurement of Noncooperative Space Targets for a Dual-Arm Space Robot , 2020, IEEE Transactions on Instrumentation and Measurement.

[30]  Chi Zhang,et al.  Iterative Distance-Aware Similarity Matrix Convolution with Mutual-Supervised Point Elimination for Efficient Point Cloud Registration , 2019, ECCV.

[31]  Kevin Gimpel,et al.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations , 2019, ICLR.

[32]  Yue Wang,et al.  PRNet: Self-Supervised Learning for Partial-to-Partial Registration , 2019, NeurIPS.

[33]  Vladlen Koltun,et al.  Fully Convolutional Geometric Features , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[34]  Howie Choset,et al.  PCRNet: Point Cloud Registration Network using PointNet Encoding , 2019, ArXiv.

[35]  Duc Thanh Nguyen,et al.  Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[36]  Yue Wang,et al.  Deep Closest Point: Learning Representations for Point Cloud Registration , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Jun Wang,et al.  Dynamic Vehicle Detection With Sparse Point Clouds Based on PE-CPD , 2019, IEEE Transactions on Intelligent Transportation Systems.

[38]  Leonidas J. Guibas,et al.  KPConv: Flexible and Deformable Convolution for Point Clouds , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[39]  Shiming Xiang,et al.  Relation-Shape Convolutional Neural Network for Point Cloud Analysis , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Slobodan Ilic,et al.  3D Local Features for Direct Pairwise Registration , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Yasuhiro Aoki,et al.  PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Alexander Kolesnikov,et al.  Revisiting Self-Supervised Visual Representation Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Lars Petersson,et al.  The Alignment of the Spheres: Globally-Optimal Spherical Mixture Alignment for Camera Pose Estimation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Yinlong Liu,et al.  Efficient Global Point Cloud Registration by Matching Rotation Invariant Features Through Translation Search , 2018, ECCV.

[45]  Jan Kautz,et al.  HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration , 2018, ECCV.

[46]  Bin Li,et al.  Fully Connected Network-Based Intra Prediction for Image Coding , 2018, IEEE Transactions on Image Processing.

[47]  Bing Li,et al.  Collaborative Mapping and Autonomous Parking for Multi-Story Parking Garage , 2018, IEEE Transactions on Intelligent Transportation Systems.

[48]  Nikos Komodakis,et al.  Unsupervised Representation Learning by Predicting Image Rotations , 2018, ICLR.

[49]  Dong Tian,et al.  FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[50]  Leonidas J. Guibas,et al.  Learning Representations and Generative Models for 3D Point Clouds , 2017, ICML.

[51]  Bolei Zhou,et al.  SegICP: Integrated deep semantic segmentation and pose estimation , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[52]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Matthias Nießner,et al.  3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Vladlen Koltun,et al.  Fast Global Registration , 2016, ECCV.

[56]  Jiaolong Yang,et al.  Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Lars Petersson,et al.  GOGMA: Globally-Optimal Gaussian Mixture Alignment , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Mohammed Bennamoun,et al.  An Integrated Framework for 3-D Modeling, Object Detection, and Pose Estimation From Point-Clouds , 2015, IEEE Transactions on Instrumentation and Measurement.

[59]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Andrea Tagliasacchi,et al.  Sparse Iterative Closest Point , 2013, Comput. Graph. Forum.

[61]  Andrew W. Fitzgibbon,et al.  Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Sivaraman Balakrishnan,et al.  Optimal kernel choice for large-scale two-sample tests , 2012, NIPS.

[63]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  脇元 修一,et al.  IEEE International Conference on Robotics and Automation (ICRA) におけるフルードパワー技術の研究動向 , 2011 .

[65]  Baba C. Vemuri,et al.  Robust Point Set Registration Using Gaussian Mixture Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Richard Szeliski,et al.  Building Rome in a day , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[67]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[68]  Aleksandr V. Segal,et al.  Generalized-ICP , 2009, Robotics: Science and Systems.

[69]  Nico Blodow,et al.  Aligning point cloud views using persistent feature histograms , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[70]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[71]  Pavel Krsek,et al.  The Trimmed Iterative Closest Point algorithm , 2002, Object recognition supported by user interaction for service robots.

[72]  Andrew W. Fitzgibbon,et al.  Robust Registration of 2D and 3D Point Sets , 2003, BMVC.

[73]  T. Peters,et al.  An algorithmic overview of surface registration techniques for medical imaging , 2000, Medical Image Anal..

[74]  Yehoshua Y. Zeevi,et al.  The farthest point strategy for progressive image sampling , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 2 - Conference B: Computer Vision & Image Processing. (Cat. No.94CH3440-5).

[75]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[76]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[78]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[79]  Michael Ian Shamos,et al.  Closest-point problems , 1975, 16th Annual Symposium on Foundations of Computer Science (sfcs 1975).

[80]  Richard Sinkhorn A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .