A Performance Evaluation of Correspondence Grouping Methods for 3D Rigid Data Matching

Seeking consistent point-to-point correspondences between 3D rigid data (point clouds, meshes, or depth maps) is a fundamental problem in 3D computer vision. While a number of correspondence selection methods have been proposed in recent years, their advantages and shortcomings remain unclear regarding different applications and perturbations. To fill this gap, this paper gives a comprehensive evaluation of nine state-of-the-art 3D correspondence grouping methods. A good correspondence grouping algorithm is expected to retrieve as many as inliers from initial feature matches, giving a rise in both precision and recall as well as facilitating accurate transformation estimation. Toward this rule, we deploy experiments on three benchmarks with different application contexts, including shape retrieval, 3D object recognition, and point cloud registration. We also investigate various perturbations such as noise, point density variation, clutter, occlusion, partial overlap, different scales of initial correspondences, and different combinations of keypoint detectors and descriptors. The rich variety of application scenarios and nuisances result in different spatial distributions and inlier ratios of initial feature correspondences, thus enabling a thorough evaluation. Based on the outcomes, we give a summary of the traits, merits, and demerits of evaluated approaches and indicate some potential future research directions.

[1]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[2]  Hui Chen,et al.  3D free-form object recognition in range images using local surface patches , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[3]  Henrik Gordon Petersen,et al.  In Search of Inliers: 3D Correspondence by Local and Global Voting , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  D. M. V. Hesteren Evolutionary Game Theory , 2017 .

[5]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[6]  Junjun Jiang,et al.  Locality Preserving Matching , 2017, IJCAI.

[7]  Yu Zhong,et al.  Intrinsic shape signatures: A shape descriptor for 3D object recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[8]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[9]  Ian D. Reid,et al.  MatchBench: An Evaluation of Feature Matchers , 2018, ArXiv.

[10]  Marc Alexa,et al.  Recent Advances in Mesh Morphing , 2002, Comput. Graph. Forum.

[11]  Dirk Kraft,et al.  Local Point Pair Feature Histogram for Accurate 3D Matching , 2018, BMVC.

[12]  Yasuo Kuniyoshi,et al.  Elastic Net Constraints for Shape Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[13]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[14]  Benjamin Bustos,et al.  Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes , 2011, The Visual Computer.

[15]  Zhiguo Cao,et al.  Toward the Repeatability and Robustness of the Local Reference Frame for 3D Shape Matching: An Evaluation , 2018, IEEE Transactions on Image Processing.

[16]  Federico Tombari,et al.  SHOT: Unique signatures of histograms for surface and texture description , 2014, Comput. Vis. Image Underst..

[17]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[18]  Mohammed Bennamoun,et al.  A Comprehensive Performance Evaluation of 3D Local Feature Descriptors , 2015, International Journal of Computer Vision.

[19]  Nico Blodow,et al.  Persistent Point Feature Histograms for 3D Point Clouds , 2008 .

[20]  Olaf Hellwich,et al.  Comparison of 3D interest point detectors and descriptors for point cloud fusion , 2014, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[21]  Norbert Krüger,et al.  Local shape feature fusion for improved matching, pose estimation and 3D object recognition , 2016, SpringerPlus.

[22]  Federico Tombari,et al.  Performance Evaluation of 3D Keypoint Detectors , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[23]  Nico Blodow,et al.  Aligning point cloud views using persistent feature histograms , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[25]  Mohammed Bennamoun,et al.  Rotational Projection Statistics for 3D Local Surface Description and Object Recognition , 2013, International Journal of Computer Vision.

[26]  Luigi di Stefano,et al.  A Repeatable and Efficient Canonical Reference for Surface Matching , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[27]  Tat-Jun Chin,et al.  Guaranteed Outlier Removal for Point Cloud Registration with Correspondences , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Lance R. Williams,et al.  Segmentation of Multiple Salient Closed Contours from Real Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[30]  Federico Tombari,et al.  Object Recognition in 3D Scenes with Occlusions and Clutter by Hough Voting , 2010, 2010 Fourth Pacific-Rim Symposium on Image and Video Technology.

[31]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Adrian Hilton,et al.  Evaluation of 3D Feature Descriptors for Multi-modal Data Registration , 2013, 2013 International Conference on 3D Vision.

[33]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Mohammed Bennamoun,et al.  A Novel Representation and Feature Matching Algorithm for Automatic Pairwise Registration of Range Images , 2005, International Journal of Computer Vision.

[35]  Mohammed Bennamoun,et al.  On the Repeatability and Quality of Keypoints for Local Feature-based 3D Object Retrieval from Cluttered Scenes , 2009, International Journal of Computer Vision.

[36]  Nassir Navab,et al.  When 2.5D is not enough: Simultaneous reconstruction, segmentation and recognition on dense SLAM , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[37]  Andrew E. Johnson,et al.  Surface matching for object recognition in complex three-dimensional scenes , 1998, Image Vis. Comput..

[38]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[39]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[40]  Weidong Yang,et al.  Ranking 3D feature correspondences via consistency voting , 2019, Pattern Recognit. Lett..

[41]  Zhiguo Cao,et al.  A fast and robust local descriptor for 3D point cloud registration , 2016, Inf. Sci..

[42]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[43]  Federico Tombari,et al.  On the Affinity between 3D Detectors and Descriptors , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[44]  Mohammed Bennamoun,et al.  Automatic Correspondence for 3d Modeling: an Extensive Review , 2005, Int. J. Shape Model..

[45]  Mohammed Bennamoun,et al.  Three-Dimensional Model-Based Object Recognition and Segmentation in Cluttered Scenes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Mohammed Bennamoun,et al.  3D Object Recognition in Cluttered Scenes with Local Surface Features: A Survey , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Yasuyuki Matsushita,et al.  GMS: Grid-Based Motion Statistics for Fast, Ultra-robust Feature Correspondence , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[49]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[52]  Zhiguo Cao,et al.  The effect of spatial information characterization on 3D local feature descriptors: A quantitative evaluation , 2017, Pattern Recognit..

[53]  Jan-Michael Frahm,et al.  USAC: A Universal Framework for Random Sample Consensus , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Radu Horaud,et al.  SHREC '11: Robust Feature Detection and Description Benchmark , 2011, 3DOR@Eurographics.

[55]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[56]  Andrea Torsello,et al.  A Scale Independent Selection Process for 3D Object Recognition in Cluttered Scenes , 2013, International Journal of Computer Vision.

[57]  Zhiguo Cao,et al.  Performance Evaluation of 3D Correspondence Grouping Algorithms , 2017, 2017 International Conference on 3D Vision (3DV).

[58]  Zhiguo Cao,et al.  Rotational contour signatures for both real-valued and binary feature representations of 3D local shape , 2017, Comput. Vis. Image Underst..

[59]  Jan-Michael Frahm,et al.  A Comparative Analysis of RANSAC Techniques Leading to Adaptive Real-Time Random Sample Consensus , 2008, ECCV.

[60]  Luigi di Stefano,et al.  Pairwise Registration by Local Orientation Cues , 2016, Comput. Graph. Forum.

[61]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[62]  Paul F. Whelan,et al.  Comparing 3D Descriptors for Local Search of Craniofacial Landmarks , 2012, ISVC.

[63]  Luigi di Stefano,et al.  On the repeatability of the local reference frame for partial shape matching , 2011, 2011 International Conference on Computer Vision.