Adaptive Fusion-Based 3D Keypoint Detection for RGB Point Clouds

We propose a novel keypoint detector for 3D RGB Point Clouds (PCs). The proposed keypoint detector exploits both the 3D structure and the RGB information of the PC data. Keypoint candidates are generated by computing the eigenvalues of the covariance matrix of the PC structure information. Additionally, from the RGB information, we estimate the salient points by an efficient adaptive difference of Gaussian-based operator. Finally, we fuse the resulting two sets of salient points to improve the repeatability of the 3D keypoint detector. The proposed algorithm is compared against the state-of-the-art algorithms on two benchmark datasets. The experimental results show that the proposed scheme outperforms the best existing method by 5.35% and 60.98 points on the SHOT-Kinect dataset and by 5.45% and 145.54 points on the SHOT-SpaceTime dataset in terms of relative and absolute repeatability, respectively.

[1]  Martial Hebert,et al.  Multi-scale interest regions from unorganized point clouds , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[2]  Umberto Castellani,et al.  Sparse points matching by combining 3D mesh saliency with statistical descriptors , 2008, Comput. Graph. Forum.

[3]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Ziyan Wu,et al.  End-to-End Learning of Keypoint Detector and Descriptor for Pose Invariant 3D Matching , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Weisi Lin,et al.  Detecting keypoint sets on 3D point clouds via Histogram of Normal Orientations , 2016, Pattern Recognit. Lett..

[6]  Yu Zhong,et al.  Intrinsic shape signatures: A shape descriptor for 3D object recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[7]  Mohammed Bennamoun,et al.  On the Repeatability and Quality of Keypoints for Local Feature-based 3D Object Retrieval from Cluttered Scenes , 2009, International Journal of Computer Vision.

[8]  Paul Checchin,et al.  Performance evaluation of 3D keypoint detectors for time-of-flight depth data , 2016, 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV).

[9]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[10]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.

[11]  Benjamin Bustos,et al.  Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes , 2011, The Visual Computer.

[12]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Hui Chen,et al.  3D free-form object recognition in range images using local surface patches , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[14]  Federico Tombari,et al.  Learning to Detect Good 3D Keypoints , 2017, International Journal of Computer Vision.

[15]  Federico Tombari,et al.  Learning a Descriptor-Specific 3D Keypoint Detector , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[17]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[18]  Federico Tombari,et al.  Performance Evaluation of 3D Keypoint Detectors , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[19]  Federico Tombari,et al.  Object Recognition in 3D Scenes with Occlusions and Clutter by Hough Voting , 2010, 2010 Fourth Pacific-Rim Symposium on Image and Video Technology.