Latent Space Representation for Shape Analysis and Learning

We propose a novel shape representation useful for analyzing and processing shape collections, as well for a variety of learning and inference tasks. Unlike most approaches that capture variability in a collection by using a template model or a base shape, we show that it is possible to construct a full shape representation by using the latent space induced by a functional map net- work, allowing us to represent shapes in the context of a collection without the bias induced by selecting a template shape. Key to our construction is a novel analysis of latent functional spaces, which shows that after proper regularization they can be endowed with a natural geometric structure, giving rise to a well-defined, stable and fully informative shape representation. We demonstrate the utility of our representation in shape analysis tasks, such as highlighting the most distorted shape parts in a collection or separating variability modes between shape classes. We further exploit our representation in learning applications by showing how it can naturally be used within deep learning and convolutional neural networks for shape classi cation or reconstruction, signi cantly outperforming existing point-based techniques.

[1]  D'arcy W. Thompson.  On growth and form i , 1943 .

[2]  D. Kendall.  A Survey of the Statistical Theory of Shape , 1989 .

[3]  U. Grenander,et al.  Computational anatomy: an emerging discipline , 1998 .

[4]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[5]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[6]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[7]  Guido Gerig,et al.  Unbiased diffeomorphic atlas construction for computational anatomy , 2004, NeuroImage.

[8]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[9]  Niklas Peinecke,et al.  Laplace-Beltrami spectra as 'Shape-DNA' of surfaces and solids , 2006, Comput. Aided Des..

[10]  Leonidas J. Guibas,et al.  Eurographics Symposium on Geometry Processing (2007) Reconstruction of Deforming Geometry from Time-varying Point Clouds , 2022 .

[11]  Hans-Peter Seidel,et al.  Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data , 2009, TOGS.

[12]  Hans-Peter Seidel,et al.  A Statistical Model of Human Pose and Body Shape , 2009, Comput. Graph. Forum.

[13]  L. Younes.  Shapes and Diffeomorphisms , 2010 .

[14]  N. Mitra,et al.  Exploration of continuous variability in collections of 3D shapes , 2011, SIGGRAPH 2011.

[15]  Stephen DiVerdi,et al.  Exploring collections of 3D models using fuzzy correspondences , 2012, ACM Trans. Graph..

[16]  Maks Ovsjanikov,et al.  Functional maps , 2012, ACM Trans. Graph..

[17]  Amit Singer,et al.  Exact and Stable Recovery of Rotations for Robust Synchronization , 2012, ArXiv.

[18]  Leonidas J. Guibas,et al.  Soft Maps Between Surfaces , 2012, Comput. Graph. Forum.

[19]  Wei Zeng,et al.  Discrete heat kernel determines discrete Riemannian metric , 2012, Graph. Model..

[20]  Daniel Cohen-Or,et al.  Active co-analysis of a set of shapes , 2012, ACM Trans. Graph..

[21]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[22]  Alexander M. Bronstein,et al.  Coupled quasi‐harmonic bases , 2012, Comput. Graph. Forum.

[23]  Stephen DiVerdi,et al.  Learning part-based templates from large collections of 3D shapes , 2013, ACM Trans. Graph..

[24]  Leonidas J. Guibas,et al.  Map-based exploration of intrinsic shape differences and variability , 2013, ACM Trans. Graph..

[25]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[26]  Leonidas J. Guibas,et al.  Image Co-segmentation via Consistent Functional Maps , 2013, 2013 IEEE International Conference on Computer Vision.

[27]  P. Gunz,et al.  Semilandmarks: a method for quantifying curves and surfaces , 2013 .

[28]  Leonidas J. Guibas,et al.  Analysis and Visualization of Maps Between Shapes , 2013, Comput. Graph. Forum.

[29]  Leonidas J. Guibas,et al.  Unsupervised Multi-class Joint Image Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Leonidas J. Guibas,et al.  Functional map networks for analyzing and exploring large shape collections , 2014, ACM Trans. Graph..

[31]  Michael J. Black,et al.  FAUST: Dataset and Evaluation for 3D Mesh Registration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Daniel Cohen-Or,et al.  SHED: shape edit distance for fine-grained shape similarity , 2015, ACM Trans. Graph..

[33]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[34]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Davide Eynard,et al.  Shape‐from‐Operator: Recovering Shapes from Intrinsic Operators , 2015, Comput. Graph. Forum.

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[38]  Abhinav Gupta,et al.  Learning a Predictable and Generative Vector Representation for Objects , 2016, ECCV.

[39]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[40]  Zhenhua Wang,et al.  Synthesizing Training Images for Boosting Human 3D Pose Estimation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[41]  Karthik Ramani,et al.  Deep Learning 3D Shape Surfaces Using Geometry Images , 2016, ECCV.

[42]  Jonathan Masci,et al.  Learning shape correspondence with anisotropic convolutional neural networks , 2016, NIPS.

[43]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[44]  Maks Ovsjanikov,et al.  Adjoint Map Representation for Shape Analysis and Matching , 2017, Comput. Graph. Forum.

[45]  Daniel Cremers,et al.  Efficient Deformable Shape Correspondence via Kernel Matching , 2017, 2017 International Conference on 3D Vision (3DV).

[46]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Leonidas J. Guibas,et al.  Computing and processing correspondences with functional maps , 2016, SIGGRAPH Courses.

[48]  Leonidas J. Guibas,et al.  GRASS: Generative Recursive Autoencoders for Shape Structures , 2017, ACM Trans. Graph..

[49]  Michael J. Black,et al.  Dynamic FAUST: Registering Human Bodies in Motion , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Yang Liu,et al.  O-CNN , 2017, ACM Trans. Graph..

[51]  Leonidas J. Guibas,et al.  Functional characterization of intrinsic and extrinsic geometry , 2017, TOGS.

[52]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Ersin Yumer,et al.  Convolutional neural networks on surfaces via seamless toric covers , 2017, ACM Trans. Graph..

[54]  Leonidas J. Guibas,et al.  Learning Representations and Generative Models for 3D Point Clouds , 2017, ICML.

[55]  Maks Ovsjanikov,et al.  On the Stability of Functional Maps and Shape Difference Operators , 2018, Comput. Graph. Forum.