Cyclic Functional Mapping: Self-supervised correspondence between non-isometric deformable shapes

We present the first utterly self-supervised network for dense correspondence mapping between non-isometric shapes. The task of alignment in non-Euclidean domains is one of the most fundamental and crucial problems in computer vision. As 3D scanners can generate highly complex and dense models, the mission of finding dense mappings between those models is vital. The novelty of our solution is based on a cyclic mapping between metric spaces, where the distance between a pair of points should remain invariant after the full cycle. As the same learnable rules that generate the point-wise descriptors apply in both directions, the network learns invariant structures without any labels while coping with non-isometric deformations. We show here state-of-the-art-results by a large margin for a variety of tasks compared to known self-supervised and supervised methods.

[1]  Alexander M. Bronstein,et al.  Affine-invariant diffusion geometry for the analysis of deformable 3D shapes , 2010, CVPR 2011.

[2]  Daniel Cremers,et al.  Anisotropic Diffusion Descriptors , 2016, Comput. Graph. Forum.

[3]  Maks Ovsjanikov,et al.  Functional maps , 2012, ACM Trans. Graph..

[4]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[5]  Ron Kimmel,et al.  Unsupervised Learning of Dense Shape Correspondence , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Craig Gotsman,et al.  Conformal Flattening by Curvature Prescription and Metric Scaling , 2008, Comput. Graph. Forum.

[7]  J A Sethian,et al.  A fast marching level set method for monotonically advancing fronts. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Leonidas J. Guibas,et al.  Functional map networks for analyzing and exploring large shape collections , 2014, ACM Trans. Graph..

[9]  Daniel Cremers,et al.  The wave kernel signature: A quantum mechanical approach to shape analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[10]  I. Daubechies,et al.  Conformal Wasserstein distances: Comparing surfaces in polynomial time , 2011, 1103.4408.

[11]  Ron Kimmel,et al.  Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Ron Kimmel,et al.  On Bending Invariant Signatures for Surfaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Raif M. Rustamov,et al.  Laplace-Beltrami eigenfunctions for deformation invariant shape representation , 2007 .

[14]  Alexander M. Bronstein,et al.  Multigrid multidimensional scaling , 2006, Numer. Linear Algebra Appl..

[15]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Leonidas J. Guibas,et al.  Map-based exploration of intrinsic shape differences and variability , 2013, ACM Trans. Graph..

[17]  Alexander M. Bronstein,et al.  Equi-affine Invariant Geometry for Shape Analysis , 2013, Journal of Mathematical Imaging and Vision.

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[20]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[21]  Ron Kimmel,et al.  Scale Invariant Geometry for Nonrigid Shapes , 2013, SIAM J. Imaging Sci..

[22]  Hans-Peter Seidel,et al.  Intrinsic Shape Matching by Planned Landmark Sampling , 2011, Comput. Graph. Forum.

[23]  Ron Kimmel,et al.  Spectral Generalized Multi-dimensional Scaling , 2013, International Journal of Computer Vision.

[24]  Alexander M. Bronstein,et al.  Fully Spectral Partial Shape Matching , 2017, Comput. Graph. Forum.

[25]  Daniel Cremers,et al.  Efficient Deformable Shape Correspondence via Kernel Matching , 2017, 2017 International Conference on 3D Vision (3DV).

[26]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[27]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[28]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[29]  Mathieu Aubry,et al.  3D-CODED: 3D Correspondences by Deep Deformation , 2018, ECCV.

[30]  Ron Kimmel,et al.  Affine Invariant Geometry for Non-rigid Shapes , 2014, International Journal of Computer Vision.

[31]  Alexander M. Bronstein,et al.  Deep Functional Maps: Structured Prediction for Dense Shape Correspondence , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[32]  Ron Kimmel,et al.  Hierarchical Matching of Non-rigid Shapes , 2011, SSVM.

[33]  Vladlen Koltun,et al.  Robust Nonrigid Registration by Convex Optimization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[34]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[35]  Daniel Cremers,et al.  Dense Non-rigid Shape Correspondence Using Random Forests , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  A. Bronstein,et al.  Learning Spectral Descriptors for Deformable Shape Correspondence , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH '05.

[38]  Iasonas Kokkinos,et al.  Scale-invariant heat kernel signatures for non-rigid shape recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Ramesh Raskar,et al.  Scale Invariant Metrics of Volumetric Datasets , 2015, SIAM J. Imaging Sci..

[41]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[42]  Johannes Wallner,et al.  Integral invariants for robust geometry processing , 2009, Comput. Aided Geom. Des..

[43]  Adrian Hilton,et al.  Spherical matching for temporal correspondence of non-rigid surfaces , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[44]  Daniel Cremers,et al.  Partial Functional Correspondence , 2017 .

[45]  Maks Ovsjanikov,et al.  Unsupervised Deep Learning for Structured Shape Matching , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  Vladimir G. Kim,et al.  Blended intrinsic maps , 2011, ACM Trans. Graph..

[47]  Daniel Cremers,et al.  Product Manifold Filter: Non-rigid Shape Correspondence via Kernel Density Estimation in the Product Space , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Y. Aflalo,et al.  On convex relaxation of graph isomorphism , 2015, Proceedings of the National Academy of Sciences.

[49]  Yaser Sheikh,et al.  LBS Autoencoder: Self-Supervised Fitting of Articulated Meshes to Point Clouds , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  R. Kimmel,et al.  Hierarchical Framework for Shape Correspondence , 2013 .

[51]  Michael J. Black,et al.  FAUST: Dataset and Evaluation for 3D Mesh Registration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Leonidas J. Guibas,et al.  Consistent Shape Maps via Semidefinite Programming , 2013, SGP '13.