Modular Latent Spaces for Shape Correspondences

We consider the problem of transporting shape descriptors across shapes in a collection in a modular fashion, in order to establish correspondences between them. A common goal when mapping between multiple shapes is consistency, namely that compositions of maps along a cycle of shapes should be approximately an identity map. Existing attempts to enforce consistency typically require recomputing correspondences whenever a new shape is added to the collection, which can quickly become intractable. Instead, we propose an approach that is fully modular, where the bulk of the computation is done on each shape independently. To achieve this, we use intermediate nonlinear embedding spaces, computed individually on every shape; the embedding functions use ideas from diffusion geometry and capture how different descriptors on the same shape inter‐relate. We then establish linear mappings between the different embedding spaces, via a shared latent space. The introduction of nonlinear embeddings allows for more nuanced correspondences, while the modularity of the construction allows for parallelizable calculation and efficient addition of new shapes. We compare the performance of our framework to standard functional correspondence techniques and showcase the use of this framework to simple interpolation and extrapolation tasks.

[1]  Maks Ovsjanikov,et al.  Functional maps , 2012, ACM Trans. Graph..

[2]  Daniel Cremers,et al.  Consistent Partial Matching of Shape Collections via Sparse Modeling , 2017, Comput. Graph. Forum.

[3]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[4]  Ronald R. Coifman,et al.  Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators , 2005, NIPS.

[5]  Daniel Cremers,et al.  The wave kernel signature: A quantum mechanical approach to shape analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[6]  Nancy Argüelles,et al.  Author ' s , 2008 .

[7]  Roy R. Lederman,et al.  Common Manifold Learning Using Alternating-Diffusion , 2015 .

[8]  Maks Ovsjanikov,et al.  Supervised Descriptor Learning for Non-Rigid Shape Matching , 2014, ECCV Workshops.

[9]  Ann B. Lee,et al.  Geometric diffusions as a tool for harmonic analysis and structure definition of data: multiscale methods. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Leonidas J. Guibas,et al.  Stable Region Correspondences Between Non‐Isometric Shapes , 2016, Comput. Graph. Forum.

[11]  Leonidas J. Guibas,et al.  Shape Matching via Quotient Spaces , 2013 .

[12]  Bruno Lévy,et al.  Laplace-Beltrami Eigenfunctions Towards an Algorithm That "Understands" Geometry , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[13]  Vladimir G. Kim,et al.  Blended intrinsic maps , 2011, SIGGRAPH 2011.

[14]  Leonidas J. Guibas,et al.  Consistent Shape Maps via Semidefinite Programming , 2013, SGP '13.

[15]  Thomas A. Funkhouser,et al.  Interior Distance Using Barycentric Coordinates , 2009, Comput. Graph. Forum.

[16]  Aaron Hertzmann,et al.  Learning 3D mesh segmentation and labeling , 2010, SIGGRAPH 2010.

[17]  Qi-Xing Huang,et al.  Dense Human Body Correspondences Using Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Ronald R. Coifman,et al.  Audio-Visual Group Recognition Using Diffusion Maps , 2010, IEEE Transactions on Signal Processing.

[19]  Leonidas J. Guibas,et al.  Earth mover's distances on discrete surfaces , 2014, ACM Trans. Graph..

[20]  T. Funkhouser,et al.  Möbius voting for surface correspondence , 2009, SIGGRAPH 2009.

[21]  Facundo Mémoli,et al.  Eurographics Symposium on Point-based Graphics (2007) on the Use of Gromov-hausdorff Distances for Shape Comparison , 2022 .

[22]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[23]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[24]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[25]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[26]  Wojciech Basalaj,et al.  Incremental multidimensional scaling method for database visualization , 1999, Electronic Imaging.

[27]  Vladlen Koltun,et al.  Joint shape segmentation with linear programming , 2011, ACM Trans. Graph..

[28]  Leonidas J. Guibas,et al.  Functional map networks for analyzing and exploring large shape collections , 2014, ACM Trans. Graph..

[29]  Karl Pearson F.R.S. LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[30]  Konrad Schindler,et al.  VocMatch: Efficient Multiview Correspondence for Structure from Motion , 2014, ECCV.

[31]  Ron Kimmel,et al.  Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Maks Ovsjanikov,et al.  Adjoint Map Representation for Shape Analysis and Matching , 2017, Comput. Graph. Forum.

[33]  Roi Poranne,et al.  Seamless surface mappings , 2015, ACM Trans. Graph..

[34]  Leonidas J. Guibas,et al.  One Point Isometric Matching with the Heat Kernel , 2010, Comput. Graph. Forum.

[35]  Thomas A. Funkhouser,et al.  A benchmark for 3D mesh segmentation , 2009, ACM Trans. Graph..

[36]  Mauro R. Ruggeri,et al.  Spectral-Driven Isometry-Invariant Matching of 3D Shapes , 2010, International Journal of Computer Vision.

[37]  Leonidas J. Guibas,et al.  Soft Maps Between Surfaces , 2012, Comput. Graph. Forum.

[38]  Daniel Cremers,et al.  Partial Functional Correspondence , 2017 .

[39]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[40]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[41]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[42]  Davide Eynard,et al.  Coupled Functional Maps , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[43]  Leonidas J. Guibas,et al.  Image Co-segmentation via Consistent Functional Maps , 2013, 2013 IEEE International Conference on Computer Vision.

[44]  Maks Ovsjanikov,et al.  Region-Based Correspondence Between 3D Shapes via Spatially Smooth Biclustering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  Gary K. L. Tam,et al.  Registration of 3D Point Clouds and Meshes: A Survey from Rigid to Nonrigid , 2013, IEEE Transactions on Visualization and Computer Graphics.

[46]  Peter Fritzson,et al.  Modeling and Applications , 2004 .