Towards using covariance matrix pyramids as salient point descriptors in 3D point clouds

Abstract In this work, a novel salient point descriptor for 3D point clouds, called Covariance Matrix Pyramids (CMPs), is presented. With CMPs it is possible to compare unstructured and unequal numbers of points which is an important characteristic when working with point clouds. Corresponding points from different scans are matched in a pyramidal approach combined with Particle Swarm Optimization. The flexibility of CMPs is demonstrated on the basis of several databases with objects, such as 3D faces, 3D apples, 3D kitchen scenes, 3D human–machine interaction gesture sequences, and 3D buildings all recorded with different 3D sensors. Quantitative results are given and compared with other state-of-the-art descriptors, whereby CMPs show promising performance.

[1]  Wei Li,et al.  Fully affine invariant SURF for image matching , 2012, Neurocomputing.

[2]  Bogdan Kwolek,et al.  Registration of 3D facial surfaces using covariance matrix pyramids , 2010, 2010 IEEE International Conference on Robotics and Automation.

[3]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[5]  Melvyn L. Smith,et al.  3D face reconstructions from photometric stereo using near infrared and visible light , 2010, Comput. Vis. Image Underst..

[6]  Xiaopeng Zhang,et al.  Enhancing low light images using near infrared flash images , 2010, 2010 IEEE International Conference on Image Processing.

[7]  Roberto Cipolla,et al.  Segmentation and Recognition Using Structure from Motion Point Clouds , 2008, ECCV.

[8]  Fatih Murat Porikli,et al.  Region Covariance: A Fast Descriptor for Detection and Classification , 2006, ECCV.

[9]  Xuelong Li,et al.  Gabor-Based Region Covariance Matrices for Face Recognition , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Shiguang Shan,et al.  Sigma Set: A small second order statistical region descriptor , 2009, CVPR.

[12]  Gerhard Rigoll,et al.  Selecting Features in On-Line Handwritten Whiteboard Note Recognition: SFS or SFFS? , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[13]  J.N.L. Brummer,et al.  An Euclidean distance measure between covariance matrices of speech cepstra for text-independent speaker recognition , 1997, Proceedings of the 1997 South African Symposium on Communications and Signal Processing. COMSIG '97.

[14]  Kongqiao Wang,et al.  Robust CoHOG Feature Extraction in Human-Centered Image/Video Management System , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  LoTsz-Wai Rachel,et al.  Local feature extraction and matching on range images , 2009 .

[16]  N. Ayache,et al.  Log‐Euclidean metrics for fast and simple calculus on diffusion tensors , 2006, Magnetic resonance in medicine.

[17]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[18]  Ying Wu,et al.  Sparsity model for robust optical flow estimation at motion discontinuities , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Bruno Lévy,et al.  ABF++: fast and robust angle based flattening , 2005, TOGS.

[20]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[21]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[22]  Yasuyuki Matsushita,et al.  Motion detail preserving optical flow estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Kostas Daniilidis,et al.  Fully Automatic Registration of 3D Point Clouds , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[25]  J. Paul Siebert,et al.  Local feature extraction and matching on range images: 2.5D SIFT , 2009, Comput. Vis. Image Underst..

[26]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Nico Blodow,et al.  Persistent Point Feature Histograms for 3D Point Clouds , 2008 .

[28]  Xuelong Li,et al.  Effective Feature Extraction in High-Dimensional Space , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Jing Pan,et al.  Scale invariant image matching using triplewise constraint and weighted voting , 2012, Neurocomputing.

[30]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[31]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[32]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[33]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[35]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[36]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[37]  Russell C. Eberhart,et al.  Solving Constrained Nonlinear Optimization Problems with Particle Swarm Optimization , 2002 .

[38]  Bruno Lévy,et al.  Least squares conformal maps for automatic texture atlas generation , 2002, ACM Trans. Graph..

[39]  Sen Wang,et al.  High resolution tracking of non-rigid 3D motion of densely sampled data using harmonic maps , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.