Spatially aggregating spectral descriptors for nonrigid 3D shape retrieval: a comparative survey

This paper presents a comprehensive review and analysis of recent spectral shape descriptors for nonrigid 3D shape retrieval. More specifically, we compare the latest spectral descriptors based on the Laplace–Beltrami (LB) operator, including ShapeDNA, heat kernel signature, scale invariant heat kernel signature, heat mean signature, wave kernel signature, and global point signature. We also include the eigenvalue descriptor (EVD), which is a geodesic distance-based shape signature. The global descriptors ShapeDNA and EVD are compared via the chi-squared distance, while all local descriptors are compared using the codebook model. Moreover, we investigate the ambiguity modeling of codebook for the densely distributed low-level shape descriptors. Inspired by the ability of spatial cues to improve discrimination between shapes, we also propose to adopt the isocontours of the second eigenfunction of the LB operator to perform surface partition, which can significantly ameliorate the retrieval performance of the time-scaled local descriptors. In addition, we introduce an intrinsic spatial pyramid matching approach in a bid to further enhance the retrieval accuracy. Extensive experiments are carried out on two 3D shape benchmarks to assess the performance of the spectral descriptors. Our proposed approach is shown to provide the best performance.

[1]  Paul Suetens,et al.  SHREC'10 Track: Non-rigid 3D Shape Retrieval , 2010, 3DOR@Eurographics.

[2]  Bruno Lévy,et al.  Laplace-Beltrami Eigenfunctions Towards an Algorithm That "Understands" Geometry , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[3]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[4]  Shawn D. Newsam,et al.  Spatial pyramid co-occurrence for image classification , 2011, 2011 International Conference on Computer Vision.

[5]  Niklas Peinecke,et al.  Laplace-Beltrami spectra as 'Shape-DNA' of surfaces and solids , 2006, Comput. Aided Des..

[6]  Eli Shechtman,et al.  In defense of Nearest-Neighbor based image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[8]  Zhang Yao,et al.  Content-Based 3-D Model Retrieval: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Ali Shokoufandeh,et al.  Retrieving articulated 3-D models using medial surfaces , 2008, Machine Vision and Applications.

[10]  Silvio Savarese,et al.  Discriminative Object Class Models of Appearance and Shape by Correlatons , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Hao Zhang,et al.  A spectral approach to shape-based retrieval of articulated 3D models , 2007, Comput. Aided Des..

[12]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[13]  Arthur W. Toga,et al.  Anisotropic Laplace-Beltrami eigenmaps: Bridging Reeb graphs and skeletons , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[14]  Alexander M. Bronstein,et al.  Shape Recognition with Spectral Distances , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Kaleem Siddiqi,et al.  Medial Representations: Mathematics, Algorithms and Applications , 2008 .

[16]  A. Yuille,et al.  Dense Scale Invariant Descriptors for Images and Surfaces , 2012 .

[17]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR '00.

[18]  Alberto Del Bimbo,et al.  Content-based retrieval of 3D models , 2006, TOMCCAP.

[19]  Frédéric Jurie,et al.  Modeling spatial layout with fisher vectors for image categorization , 2011, 2011 International Conference on Computer Vision.

[20]  S. Rosenberg The Laplacian on a Riemannian Manifold: The Laplacian on a Riemannian Manifold , 1997 .

[21]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Aly A. Farag,et al.  CSIFT: A SIFT Descriptor with Color Invariant Characteristics , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[24]  Raif M. Rustamov,et al.  Laplace-Beltrami eigenfunctions for deformation invariant shape representation , 2007 .

[25]  Karthik Ramani,et al.  Heat-mapping: A robust approach toward perceptually consistent mesh segmentation , 2011, CVPR 2011.

[26]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Tsuhan Chen,et al.  Image retrieval with geometry-preserving visual phrases , 2011, CVPR 2011.

[28]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[29]  Paul Suetens,et al.  SHREC '11 Track: Shape Retrieval on Non-rigid 3D Watertight Meshes , 2011, 3DOR@Eurographics.

[30]  Remco C. Veltkamp,et al.  A Survey of Content Based 3D Shape Retrieval Methods , 2004, SMI.

[31]  Ramsay Dyer,et al.  Spectral Mesh Processing , 2010, Comput. Graph. Forum.

[32]  Craig Gotsman,et al.  A multi-resolution approach to heat kernels on discrete surfaces , 2010, ACM Trans. Graph..

[33]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[34]  François Fouss,et al.  Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative Recommendation , 2007, IEEE Transactions on Knowledge and Data Engineering.

[35]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Changhu Wang,et al.  Spatial-bag-of-features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Ron Kimmel,et al.  On Bending Invariant Signatures for Surfaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Alexander M. Bronstein,et al.  Spatially-Sensitive Affine-Invariant Image Descriptors , 2010, ECCV.

[39]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[40]  David Picard,et al.  Improving image similarity with vectors of locally aggregated tensors , 2011, 2011 18th IEEE International Conference on Image Processing.

[41]  Daniel Cremers,et al.  The wave kernel signature: A quantum mechanical approach to shape analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[42]  Karen K. Uhlenbeck Generic Properties of Eigenfunctions , 1976 .

[43]  A. Ben Hamza,et al.  Skeleton Path Based Approach for Nonrigid 3D Shape Analysis and Retrieval , 2011, IWCIA.

[44]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[45]  Hao Zhang,et al.  Non-Rigid Spectral Correspondence of Triangle Meshes , 2007, Int. J. Shape Model..

[46]  Mikhail Belkin,et al.  Discrete laplace operator on meshed surfaces , 2008, SCG '08.

[47]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[48]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Konrad Polthier,et al.  On approximation of the Laplace–Beltrami operator and the Willmore energy of surfaces , 2011, Comput. Graph. Forum.

[50]  Stefano Soatto,et al.  Proximity Distribution Kernels for Geometric Context in Category Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[51]  Thomas A. Funkhouser,et al.  Biharmonic distance , 2010, TOGS.

[52]  Marcel Körtgen,et al.  3D Shape Matching with 3D Shape Contexts , 2003 .

[53]  Florent Perronnin,et al.  Fisher Kernels on Visual Vocabularies for Image Categorization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Trevor Darrell,et al.  Beyond spatial pyramids: Receptive field learning for pooled image features , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Nikos Paragios,et al.  Graph commute times for image representation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Trevor Darrell,et al.  The pyramid match kernel: discriminative classification with sets of image features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[57]  Cor J. Veenman,et al.  Visual Word Ambiguity , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[59]  Eitan Grinspun,et al.  Discrete laplace operators: no free lunch , 2007, Symposium on Geometry Processing.

[60]  Gang Hua,et al.  Integrated feature selection and higher-order spatial feature extraction for object categorization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Mohamed Daoudi,et al.  Indexed heat curves for 3D-model retrieval , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[62]  Cor J. Veenman,et al.  Comparing compact codebooks for visual categorization , 2010, Comput. Vis. Image Underst..

[63]  BENJAMIN BUSTOS,et al.  Feature-based similarity search in 3D object databases , 2005, CSUR.

[64]  A. Ben Hamza,et al.  Geodesic matching of triangulated surfaces , 2006, IEEE Transactions on Image Processing.

[65]  Yuri Safarov,et al.  Spectral Theory and Geometry , 1999 .

[66]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[67]  Sven J. Dickinson,et al.  From skeletons to bone graphs: Medial abstraction for object recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[68]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[69]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[70]  Karthik Ramani,et al.  Temperature distribution descriptor for robust 3D shape retrieval , 2011, CVPR 2011 WORKSHOPS.

[71]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[72]  A. Ben Hamza,et al.  Reeb graph path dissimilarity for 3D object matching and retrieval , 2011, The Visual Computer.