Minimal Solvers for 3D Scan Alignment With Pairs of Intersecting Lines

We explore the possibility of using line intersection constraints for 3D scan registration. Typical 3D registration algorithms exploit point and plane correspondences, while line intersection constraints have not been used in the context of 3D scan registration before. Constraints from a match of pairs of intersecting lines in two 3D scans can be seen as two 3D line intersections, a plane correspondence, and a point correspondence. In this paper, we present minimal solvers that combine these different type of constraints: 1) three line intersections and one point match; 2) one line intersection and two point matches; 3) three line intersections and one plane match; 4) one line intersection and two plane matches; and 5) one line intersection, one point match, and one plane match. To use all the available solvers, we present a hybrid RANSAC loop. We propose a non-linear refinement technique using all the inliers obtained from the RANSAC. Vast experiments with simulated data and two real-data data-sets show that the use of these features and the combined solvers improve the accuracy. The code is available.

[1]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[2]  Miguel Á. Carreira-Perpiñán,et al.  Non-rigid point set registration: Coherent Point Drift , 2006, NIPS.

[3]  Marc Pollefeys,et al.  Minimal solutions for the multi-camera pose estimation problem , 2015, Int. J. Robotics Res..

[4]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Leah Blau,et al.  Computational Line Geometry , 2016 .

[6]  Andrew Wilson,et al.  MirageTable: freehand interaction on a projected augmented reality tabletop , 2012, CHI.

[7]  Carl Olsson,et al.  Branch-and-Bound Methods for Euclidean Registration Problems , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Carolina Raposo,et al.  3D Registration of Curves and Surfaces Using Local Differential Information , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Karl Johan Åström,et al.  Solutions to Minimal Generalized Relative Pose Problems , 2005 .

[10]  Forstner Wolfgang,et al.  Efficient and Accurate Registration of Point Clouds with Plane to Plane Correspondences , 2017 .

[11]  Yasuhiro Aoki,et al.  PointNetLK: Robust & Efficient Point Cloud Registration Using PointNet , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Wolfram Burgard,et al.  3-D Mapping With an RGB-D Camera , 2014, IEEE Transactions on Robotics.

[13]  J BeslPaul,et al.  A Method for Registration of 3-D Shapes , 1992 .

[14]  Yan Lu,et al.  Robust RGB-D Odometry Using Point and Line Features , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Andrew Owens,et al.  SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Zi Jian Yew,et al.  3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration , 2018, ECCV.

[17]  Vladlen Koltun,et al.  Open3D: A Modern Library for 3D Data Processing , 2018, ArXiv.

[18]  Andrew W. Fitzgibbon Robust registration of 2D and 3D point sets , 2003, Image Vis. Comput..

[19]  Pavel Krsek,et al.  The Trimmed Iterative Closest Point algorithm , 2002, Object recognition supported by user interaction for service robots.

[20]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[21]  Michel Antunes,et al.  Plane-based Odometry using an RGB-D Camera , 2013, BMVC.

[22]  John J. Leonard,et al.  Deformation-based loop closure for large scale dense RGB-D SLAM , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Niloy J. Mitra,et al.  Super4PCS: Fast Global Pointcloud Registration via Smart Indexing , 2019 .

[24]  P. Schönemann,et al.  A generalized solution of the orthogonal procrustes problem , 1966 .

[25]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[26]  Jan-Michael Frahm,et al.  Scanning and tracking dynamic objects with commodity depth cameras , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[27]  Albert S. Huang,et al.  Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera , 2011, ISRR.

[28]  Anath Fischer,et al.  3D Point Cloud Registration for Localization Using a Deep Neural Network Auto-Encoder , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[30]  Venu Madhav Govindu,et al.  Combining two-view constraints for motion estimation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  Takeo Kanade,et al.  A Correlation-Based Approach to Robust Point Set Registration , 2004, ECCV.

[32]  Shree K. Nayar,et al.  A general imaging model and a method for finding its parameters , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[33]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[34]  Chen Feng,et al.  Point-plane SLAM for hand-held 3D sensors , 2013, 2013 IEEE International Conference on Robotics and Automation.

[35]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Yuichi Taguchi,et al.  A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization , 2013, International Journal of Computer Vision.

[37]  Pedro Miraldo,et al.  A Minimal Closed-Form Solution for Multi-Perspective Pose Estimation using Points and Lines , 2018, ECCV.

[38]  Peter F. Sturm,et al.  Multi-view geometry for general camera models , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[39]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[40]  Dieter Schmalstieg,et al.  A Minimal Solution to the Generalized Pose-and-Scale Problem , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Zuzana Kukelova,et al.  Automatic Generator of Minimal Problem Solvers , 2008, ECCV.

[42]  Jan-Michael Frahm,et al.  A new minimal solution to the relative pose of a calibrated stereo camera with small field of view overlap , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[43]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[44]  Slobodan Ilic,et al.  PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors , 2018, ECCV.

[45]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[46]  Rama Chellappa,et al.  3DRegNet: A Deep Neural Network for 3D Point Registration , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Hongdong Li,et al.  The 3D-3D Registration Problem Revisited , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[48]  Chen Feng,et al.  DeepMapping: Unsupervised Map Estimation From Multiple Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Florent Lamiraux,et al.  Metric-based iterative closest point scan matching for sensor displacement estimation , 2006, IEEE Transactions on Robotics.

[50]  Vladlen Koltun,et al.  Fast Global Registration , 2016, ECCV.

[51]  Vladlen Koltun,et al.  Robust reconstruction of indoor scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[53]  Vladlen Koltun,et al.  Colored Point Cloud Registration Revisited , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[54]  Gary K. L. Tam,et al.  Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid , 2013, IEEE Transactions on Visualization and Computer Graphics.

[55]  Helder Araújo,et al.  Calibration of Smooth Camera Models , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Yue Wang,et al.  Deep Closest Point: Learning Representations for Point Cloud Registration , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[57]  Kai Li,et al.  Robust Line Matching Based on Ray-Point-Ray Structure Descriptor , 2014, ACCV Workshops.

[58]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[59]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[60]  Srikumar Ramalingam,et al.  A Unifying Model for Camera Calibration , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Daniel Cremers,et al.  Large-Scale Multi-resolution Surface Reconstruction from RGB-D Sequences , 2013, 2013 IEEE International Conference on Computer Vision.

[62]  Andrew W. Fitzgibbon,et al.  KinÊtre: animating the world with the human body , 2012, UIST.

[63]  Robert Pless,et al.  Using many cameras as one , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[64]  Daniel Cremers,et al.  Robust odometry estimation for RGB-D cameras , 2013, 2013 IEEE International Conference on Robotics and Automation.

[65]  Hongdong Li,et al.  Five-Point Motion Estimation Made Easy , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[66]  Lei Zhou,et al.  Learning and Matching Multi-View Descriptors for Registration of Point Clouds , 2018, ECCV.

[67]  Hongdong Li,et al.  Rotation Averaging , 2013, International Journal of Computer Vision.

[68]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[69]  Andrew W. Fitzgibbon,et al.  Robust Registration of 2D and 3D Point Sets , 2003, BMVC.

[70]  Slobodan Ilic,et al.  3D Local Features for Direct Pairwise Registration , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Roland Siegwart,et al.  A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation , 2011, CVPR 2011.

[72]  Torsten Sattler,et al.  Hybrid Camera Pose Estimation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[73]  Pedro Miraldo,et al.  Minimal Solvers for Mini-Loop Closures in 3D Multi-Scan Alignment , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  Shiyu Song,et al.  DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[75]  Uttaran Bhattacharya,et al.  Efficient and Robust Registration on the 3D Special Euclidean Group , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[76]  Wolfgang Förstner,et al.  Efficient and Accurate Registration of Point Clouds with Plane to Plane Correspondences , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[77]  Li Li,et al.  Hierarchical line matching based on Line-Junction-Line structure descriptor and local homography estimation , 2016, Neurocomputing.

[78]  Jiaolong Yang,et al.  Go-ICP: Solving 3D Registration Efficiently and Globally Optimally , 2013, 2013 IEEE International Conference on Computer Vision.

[79]  Zuzana Kukelova,et al.  Making Minimal Solvers for Absolute Pose Estimation Compact and Robust , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).