Unsupervised Deep Learning for Structured Shape Matching

We present a novel method for computing correspondences across 3D shapes using unsupervised learning. Our method computes a non-linear transformation of given descriptor functions, while optimizing for global structural properties of the resulting maps, such as their bijectivity or approximate isometry. To this end, we use the functional maps framework, and build upon the recent FMNet architecture for descriptor learning. Unlike that approach, however, we show that learning can be done in a purely \emph{unsupervised setting}, without having access to any ground truth correspondences. This results in a very general shape matching method that we call SURFMNet for Spectral Unsupervised FMNet, and which can be used to establish correspondences within 3D shape collections without any prior information. We demonstrate on a wide range of challenging benchmarks, that our approach leads to state-of-the-art results compared to the existing unsupervised methods and achieves results that are comparable even to the supervised learning techniques. Moreover, our framework is an order of magnitude faster, and does not rely on geodesic distance computation or expensive post-processing.

[1]  Ron Kimmel,et al.  Spectral Generalized Multi-dimensional Scaling , 2013, International Journal of Computer Vision.

[2]  Qi-Xing Huang,et al.  Dense Human Body Correspondences Using Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Alexander M. Bronstein,et al.  Fully Spectral Partial Shape Matching , 2017, Comput. Graph. Forum.

[4]  Daniel Cremers,et al.  Efficient Deformable Shape Correspondence via Kernel Matching , 2017, 2017 International Conference on 3D Vision (3DV).

[5]  Alexander M. Bronstein,et al.  Self-supervised Learning of Dense Shape Correspondence , 2018, ArXiv.

[6]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[7]  Maks Ovsjanikov,et al.  Functional maps , 2012, ACM Trans. Graph..

[8]  Xavier Bresson,et al.  Functional correspondence by matrix completion , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Jovan Popović,et al.  Deformation transfer for triangle meshes , 2004, SIGGRAPH 2004.

[10]  Andrea Torsello,et al.  Matching Deformable Objects in Clutter , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[11]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[12]  S. Rosenberg The Laplacian on a Riemannian Manifold: An Introduction to Analysis on Manifolds , 1997 .

[13]  Leonidas J. Guibas,et al.  Map-based exploration of intrinsic shape differences and variability , 2013, ACM Trans. Graph..

[14]  Michael J. Black,et al.  FAUST: Dataset and Evaluation for 3D Mesh Registration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Maks Ovsjanikov,et al.  Supervised Descriptor Learning for Non-Rigid Shape Matching , 2014, ECCV Workshops.

[16]  Maks Ovsjanikov,et al.  Multi-directional geodesic neural networks via equivariant convolution , 2018, ACM Trans. Graph..

[17]  Pierre Vandergheynst,et al.  Learning class‐specific descriptors for deformable shapes using localized spectral convolutional networks , 2015, SGP '15.

[18]  Daniel Cremers,et al.  Partial Functional Correspondence , 2017 .

[19]  Vladimir G. Kim,et al.  Blended intrinsic maps , 2011, SIGGRAPH 2011.

[20]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[21]  Alexander M. Bronstein,et al.  Coupled quasi‐harmonic bases , 2012, Comput. Graph. Forum.

[22]  Maks Ovsjanikov,et al.  Improved Functional Mappings via Product Preservation , 2018, Comput. Graph. Forum.

[23]  Alexander M. Bronstein,et al.  Recent Trends, Applications, and Perspectives in 3D Shape Similarity Assessment , 2016, Comput. Graph. Forum.

[24]  Jonathan Masci,et al.  Learning shape correspondence with anisotropic convolutional neural networks , 2016, NIPS.

[25]  Daniel Cremers,et al.  Dense Non-rigid Shape Correspondence Using Random Forests , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Maks Ovsjanikov,et al.  Adjoint Map Representation for Shape Analysis and Matching , 2017, Comput. Graph. Forum.

[27]  Leonidas J. Guibas,et al.  Computing and processing correspondences with functional maps , 2016, SIGGRAPH Courses.

[28]  Mirela Ben-Chen,et al.  Deblurring and Denoising of Maps between Shapes , 2017, Comput. Graph. Forum.

[29]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[31]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH '05.

[32]  Y. Wang,et al.  Vector Field Map Representation for Near Conformal Surface Correspondence , 2018, Comput. Graph. Forum.

[33]  Davide Eynard,et al.  Coupled Functional Maps , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[34]  Ron Kimmel,et al.  Unsupervised Learning of Dense Shape Correspondence , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Michael Möller,et al.  Point-wise Map Recovery and Refinement from Functional Correspondence , 2015, VMV.

[36]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Maks Ovsjanikov,et al.  Topological Function Optimization for Continuous Shape Matching , 2018, Comput. Graph. Forum.

[38]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[39]  Leonidas J. Guibas,et al.  Functional map networks for analyzing and exploring large shape collections , 2014, ACM Trans. Graph..

[40]  Jing Ren,et al.  Continuous and orientation-preserving correspondences via functional maps , 2018, ACM Trans. Graph..

[41]  Daniel Cremers,et al.  The wave kernel signature: A quantum mechanical approach to shape analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[42]  Dong-Ming Yan,et al.  Low-Resolution Remeshing Using the Localized Restricted Voronoi Diagram , 2014, IEEE Transactions on Visualization and Computer Graphics.

[43]  Alexander M. Bronstein,et al.  Deep Functional Maps: Structured Prediction for Dense Shape Correspondence , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[44]  Reinhard Klein,et al.  Embedding shapes with Green's functions for global shape matching , 2017, Comput. Graph..

[45]  Michael M. Bronstein,et al.  Kernel Functional Maps , 2018, Comput. Graph. Forum.

[46]  A. Bronstein,et al.  Learning Spectral Descriptors for Deformable Shape Correspondence , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[48]  R. K. Singh,et al.  Composition operators on function spaces , 1993 .

[49]  Gary K. L. Tam,et al.  Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid , 2013, IEEE Transactions on Visualization and Computer Graphics.

[50]  Daniel Cremers,et al.  Product Manifold Filter: Non-rigid Shape Correspondence via Kernel Density Estimation in the Product Space , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[52]  Maks Ovsjanikov,et al.  Informative Descriptor Preservation via Commutativity for Shape Matching , 2017, Comput. Graph. Forum.

[53]  Michael M. Bronstein,et al.  MADMM: A Generic Algorithm for Non-smooth Optimization on Manifolds , 2015, ECCV.