Spaces and manifolds of shapes in computer vision: An overview

We provide an overview of several shape space models that have been proposed in the past few years, focusing, in particular on models involving Riemannian manifolds of shapes. The discussion is organized in three stages, starting with a review of some shape representation methods, followed by shape space structures, from metric spaces to manifolds, and concluding with a short description of some of the applications that resulted from such models.

[1]  Suresh Venkatasubramanian,et al.  Robust statistics on Riemannian manifolds via the geometric median , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[3]  A. Ben Hamza,et al.  Geodesic Object Representation and Recognition , 2003, DGCI.

[4]  Michael I. Miller,et al.  Evolutions equations in computational anatomy , 2009, NeuroImage.

[5]  Rachid Deriche,et al.  Implicit Active Shape Models for 3D Segmentation in MR Imaging , 2004, MICCAI.

[6]  Stefano Soatto,et al.  DEFORMOTION: Deforming Motion, Shape Average and the Joint Registration and Segmentation of Images , 2002, ECCV.

[7]  M. McPeek,et al.  Modeling Three-Dimensional Morphological Structures Using Spherical Harmonics , 2009, Evolution; international journal of organic evolution.

[8]  Fred L. Bookstein,et al.  Shape and the information in medical images: a decade of the morphometric synthesis , 1997, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[9]  Timothy F. Cootes,et al.  Comparing Active Shape Models with Active Appearance Models , 1999, BMVC.

[10]  Olivier D. Faugeras,et al.  Approximations of Shape Metrics and Application to Shape Warping and Empirical Shape Statistics , 2005, Found. Comput. Math..

[11]  Fillia Makedon,et al.  Efficient Registration of 3D SPHARM Surfaces , 2007, Fourth Canadian Conference on Computer and Robot Vision (CRV '07).

[12]  Jun Zhang,et al.  Invariant object recognition by shape space analysis , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[13]  Ian Lewis,et al.  Proceedings of the SPIE , 2012 .

[14]  Liang Wang,et al.  Learning and Matching of Dynamic Shape Manifolds for Human Action Recognition , 2007, IEEE Transactions on Image Processing.

[15]  Alain Trouvé,et al.  Computational anatomy: computing metrics on anatomical shapes , 2002, Proceedings IEEE International Symposium on Biomedical Imaging.

[16]  Roberto Cipolla,et al.  Quasi-Invariant Parameterisations and Matching of Curves in Images , 1998, International Journal of Computer Vision.

[17]  Alain Trouvé,et al.  The Euler-Poincare theory of Metamorphosis , 2008, ArXiv.

[18]  L. Younes,et al.  Diffeomorphic matching of distributions: a new approach for unlabelled point-sets and sub-manifolds matching , 2004, CVPR 2004.

[19]  Robert Pless,et al.  On Manifold Structure of Cardiac MRI Data: Application to Segmentation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Sergey Kushnarev The geometry of the space of 2D shapes and the Weil-Petersson metric , 2010 .

[21]  Fred L. Bookstein,et al.  Landmark methods for forms without landmarks: morphometrics of group differences in outline shape , 1997, Medical Image Anal..

[22]  L. Younes,et al.  Statistics on diffeomorphisms via tangent space representations , 2004, NeuroImage.

[23]  Benjamin B. Kimia,et al.  On the Local Form and Transitions of Symmetry Sets, Medial Axes, and Shocks , 2004, International Journal of Computer Vision.

[24]  H. Le,et al.  Locating Fréchet means with application to shape spaces , 2001, Advances in Applied Probability.

[25]  Leonidas J. Guibas,et al.  Gromov‐Hausdorff Stable Signatures for Shapes using Persistence , 2009, Comput. Graph. Forum.

[26]  Stefano Soatto,et al.  Tracking deforming objects by filtering and prediction in the space of curves , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[27]  U. Grenander Elements of Pattern Theory , 1996 .

[28]  Daniel Cremers,et al.  Nonlinear Shape Statistics in Mumford-Shah Based Segmentation , 2002, ECCV.

[29]  A. Elgammal,et al.  Separating style and content on a nonlinear manifold , 2004, CVPR 2004.

[30]  Namrata Vaswani,et al.  Particle filtering for geometric active contours with application to tracking moving and deforming objects , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Guillermo Sapiro,et al.  A Theoretical and Computational Framework for Isometry Invariant Recognition of Point Cloud Data , 2005, Found. Comput. Math..

[32]  H. Le,et al.  Estimation of Riemannian Barycentres , 2004 .

[33]  James S. Duncan,et al.  Boundary Finding with Parametrically Deformable Models , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Alan L. Yuille,et al.  Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Benjamin B. Kimia,et al.  Shapes, shocks, and deformations I: The components of two-dimensional shape and the reaction-diffusion space , 1995, International Journal of Computer Vision.

[36]  Douglas W. Jones,et al.  Shape analysis of brain ventricles using SPHARM , 2001, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA 2001).

[37]  Kaleem Siddiqi,et al.  Medial Representations: Mathematics, Algorithms and Applications , 2008 .

[38]  H. Le MEAN SIZE-AND-SHAPES AND MEAN SHAPES: A GEOMETRIC POINT OF VIEW , 1995 .

[39]  Moo K. Chung,et al.  Persistence Diagrams of Cortical Surface Data , 2009, IPMI.

[40]  Peter Schröder,et al.  Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposium on Geometry processing , 2003 .

[41]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  D. Mumford,et al.  A Metric on Shape Space with Explicit Geodesics , 2007, 0706.4299.

[43]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[44]  Alain Trouvé,et al.  Metamorphoses Through Lie Group Action , 2005, Found. Comput. Math..

[45]  J. Tilak Ratnanather,et al.  Diffeomorphic Active Contours , 2010, SIAM J. Imaging Sci..

[46]  Michael I. Miller,et al.  Pattern Theory: From Representation to Inference , 2007 .

[47]  J. Tilak Ratnanather,et al.  Diffeomorphic Surface Flows: A Novel Method of Surface Evolution , 2008, SIAM J. Appl. Math..

[48]  E. Sharon,et al.  2D-Shape Analysis Using Conformal Mapping , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[49]  A. Ben Hamza,et al.  Geodesic matching of triangulated surfaces , 2006, IEEE Transactions on Image Processing.

[50]  D. Mumford,et al.  Pattern Theory: The Stochastic Analysis of Real-World Signals , 2010 .

[51]  Sergey Kushnarev,et al.  Teichons: Solitonlike Geodesics on Universal Teichmüller Space , 2009, Exp. Math..

[52]  Michael I. Miller,et al.  Transport of Relational Structures in Groups of Diffeomorphisms , 2008, Journal of Mathematical Imaging and Vision.

[53]  P. Thomas Fletcher,et al.  Population Shape Regression From Random Design Data , 2007, ICCV.

[54]  Anthony J. Yezzi,et al.  Sobolev Active Contours , 2005, International Journal of Computer Vision.

[55]  Szymon Rusinkiewicz,et al.  Proceedings of the Symposium on Geometry Processing , 2008 .

[56]  Baback Moghaddam,et al.  Principal Manifolds and Probabilistic Subspaces for Visual Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Alain Trouvé,et al.  Diffeomorphisms Groups and Pattern Matching in Image Analysis , 1998, International Journal of Computer Vision.

[58]  Jayant Shah type Riemannian metrics on the space of planar curves , 2005 .

[59]  Milan Sonka,et al.  Segmentation and interpretation of MR brain images. An improved active shape model , 1998, IEEE Transactions on Medical Imaging.

[60]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[61]  L. Younes,et al.  On the metrics and euler-lagrange equations of computational anatomy. , 2002, Annual review of biomedical engineering.

[62]  Baback Moghaddam,et al.  Principal manifolds and Bayesian subspaces for visual recognition , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[63]  Philip N. Klein,et al.  Recognition of shapes by editing their shock graphs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Alain Trouvé,et al.  Bayesian template estimation in computational anatomy , 2008, NeuroImage.

[65]  Ehud Rivlin,et al.  Semi-local invariants , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[66]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[67]  Martin Styner,et al.  Boundary and Medial Shape Analysis of the Hippocampus in Schizophrenia , 2003, MICCAI.

[68]  F. Mémoli,et al.  Fast computation of weighted distance functions and geodesics on implicit hyper-surfaces: 730 , 2001 .

[69]  Laurent Younes,et al.  Geodesic Interpolating Splines , 2001, EMMCVPR.

[70]  Anuj Srivastava,et al.  Parameterization-Invariant Shape Comparisons of Anatomical Surfaces , 2011, IEEE Transactions on Medical Imaging.

[71]  Laurent Younes,et al.  Optimal matching between shapes via elastic deformations , 1999, Image Vis. Comput..

[72]  Yunmei Chen,et al.  Using Prior Shapes in Geometric Active Contours in a Variational Framework , 2002, International Journal of Computer Vision.

[73]  Anuj Srivastava,et al.  Analysis of planar shapes using geodesic paths on shape spaces , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Ulf Grenander,et al.  Hands: A Pattern Theoretic Study of Biological Shapes , 1990 .

[75]  Laurent Younes,et al.  Geodesic Matching with Free Extremities , 2006, Journal of Mathematical Imaging and Vision.

[76]  Namrata Vaswani,et al.  Tracking Deforming Objects Using Particle Filtering for Geometric Active Contours , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Olivier D. Faugeras,et al.  Statistical shape influence in geodesic active contours , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[78]  Anuj Srivastava,et al.  On Shape of Plane Elastic Curves , 2007, International Journal of Computer Vision.

[79]  Thomas S. Huang,et al.  Image processing , 1971 .

[80]  D. Kendall A Survey of the Statistical Theory of Shape , 1989 .

[81]  D. Mumford,et al.  An overview of the Riemannian metrics on spaces of curves using the Hamiltonian approach , 2006, math/0605009.

[82]  Tosiyasu L. Kunii,et al.  Surface coding based on Morse theory , 1991, IEEE Computer Graphics and Applications.

[83]  Anthony J. Yezzi,et al.  Tracking With Sobolev Active Contours , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[84]  D. Mumford,et al.  VANISHING GEODESIC DISTANCE ON SPACES OF SUBMANIFOLDS AND DIFFEOMORPHISMS , 2004, math/0409303.

[85]  J. Damon Smoothness and geometry of boundaries associated to skeletal structures, II: Geometry in the Blum case , 2004, Compositio Mathematica.

[86]  Facundo Mémoli,et al.  Eurographics Symposium on Point-based Graphics (2007) on the Use of Gromov-hausdorff Distances for Shape Comparison , 2022 .

[87]  Yi Zhou,et al.  Bayesian tangent shape model: estimating shape and pose parameters via Bayesian inference , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[88]  Alain Trouvé,et al.  Local Geometry of Deformable Templates , 2005, SIAM J. Math. Anal..

[89]  Rama Chellappa,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Matching Shape Sequences in Video with Applications in Human Movement Analysis. Ieee Transactions on Pattern Analysis and Machine Intelligence 2 , 2022 .

[90]  Roberto Cipolla,et al.  Affine integral invariants and matching of curves , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[91]  Guillermo Sapiro,et al.  A Gromov-Hausdorff Framework with Diffusion Geometry for Topologically-Robust Non-rigid Shape Matching , 2010, International Journal of Computer Vision.

[92]  David Mumford,et al.  Shape representation via conformal mapping , 2007, Electronic Imaging.

[93]  F. Bookstein,et al.  Morphometric Tools for Landmark Data: Geometry and Biology , 1999 .

[94]  P. Thomas Fletcher,et al.  Gaussian Distributions on Lie Groups and Their Application to Statistical Shape Analysis , 2003, IPMI.

[95]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[96]  Laurent D. Cohen,et al.  On active contour models and balloons , 1991, CVGIP Image Underst..

[97]  Haibin Ling,et al.  Shape Classification Using the Inner-Distance , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[98]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[99]  Stefano Soatto,et al.  Integral Invariant Signatures , 2004, ECCV.

[100]  Anuj Srivastava,et al.  Intrinsic Bayesian Active Contours for Extraction of Object Boundaries in Images , 2009, International Journal of Computer Vision.

[101]  J. Damon Smoothness and geometry of boundaries associated to skeletal structures, II: Geometry in the Blum case , 2004, Compositio Mathematica.

[102]  Daniel Cremers,et al.  Dynamical statistical shape priors for level set-based tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[103]  D. Kendall SHAPE MANIFOLDS, PROCRUSTEAN METRICS, AND COMPLEX PROJECTIVE SPACES , 1984 .

[104]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[105]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[106]  Daniel Cremers,et al.  Integral Invariants for Shape Matching , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[107]  Jerry L. Prince,et al.  Snakes, shapes, and gradient vector flow , 1998, IEEE Trans. Image Process..

[108]  Ronen Basri,et al.  Determining the similarity of deformable shapes , 1998, Vision Research.

[109]  Daniel Cremers,et al.  Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation , 2006, International Journal of Computer Vision.

[110]  Alejandro F. Frangi,et al.  Active shape model segmentation with optimal features , 2002, IEEE Transactions on Medical Imaging.

[111]  Peter J. Olver,et al.  Joint Invariant Signatures , 2001, Found. Comput. Math..

[112]  Paul Dupuis,et al.  Variational problems on ows of di eomorphisms for image matching , 1998 .

[113]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[114]  Martin Bauer,et al.  Sobolev metrics on shape space of surfaces , 2010, 1211.3515.

[115]  Namrata Vaswani,et al.  Particle Filters for Infinite (or Large) Dimensional State Spaces-Part 2 , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[116]  James N. Damon,et al.  Determining the Geometry of Boundaries of Objects from Medial Data , 2005, International Journal of Computer Vision.

[117]  Ron Kimmel,et al.  On Bending Invariant Signatures for Surfaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[118]  Anuj Srivastava,et al.  Shape Analysis of Elastic Curves in Euclidean Spaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[119]  Franccois-Xavier Vialard,et al.  Shape Splines and Stochastic Shape Evolutions: A Second Order Point of View , 2010, 1003.3895.

[120]  Jun Ma,et al.  A Bayesian Generative Model for Surface Template Estimation , 2010, Int. J. Biomed. Imaging.

[121]  Martin Styner,et al.  Statistical shape analysis of neuroanatomical structures based on medial models , 2003, Medical Image Anal..

[122]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[123]  L. Younes Shapes and Diffeomorphisms , 2010 .

[124]  Alain Trouvé,et al.  Diffeomorphic Matching Problems in One Dimension: Designing and Minimizing Matching Functionals , 2000, ECCV.

[125]  Ehud Rivlin,et al.  Scale space semi-local invariants , 1997, Image Vis. Comput..

[126]  Yunmei Chen,et al.  On the incorporation of shape priors into geometric active contours , 2001, Proceedings IEEE Workshop on Variational and Level Set Methods in Computer Vision.

[127]  F. Bookstein,et al.  The Measurement of Biological Shape and Shape Change. , 1980 .

[128]  Alain Trouvé,et al.  Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms , 2005, International Journal of Computer Vision.

[129]  Hamid Krim,et al.  Rotation invariant topology coding of 2D and 3D objects using Morse theory , 2005, IEEE International Conference on Image Processing 2005.

[130]  P. Thomas Fletcher,et al.  Principal geodesic analysis for the study of nonlinear statistics of shape , 2004, IEEE Transactions on Medical Imaging.

[131]  Facundo Mémoli,et al.  Spectral Gromov-Wasserstein distances for shape matching , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[132]  William M. Wells,et al.  Medical Image Computing and Computer-Assisted Intervention — MICCAI’98 , 1998, Lecture Notes in Computer Science.

[133]  Paul A. Yushkevich,et al.  Multiscale deformable model segmentation and statistical shape analysis using medial descriptions , 2002, IEEE Transactions on Medical Imaging.

[134]  O. Faugeras,et al.  Level set based segmentation with intensity and curvature priors , 2000, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis. MMBIA-2000 (Cat. No.PR00737).

[135]  D. Mumford,et al.  Riemannian Geometries on Spaces of Plane Curves , 2003, math/0312384.

[136]  Michael I. Miller,et al.  Group Actions, Homeomorphisms, and Matching: A General Framework , 2004, International Journal of Computer Vision.

[137]  P. Thomas Fletcher,et al.  Statistics of shape via principal geodesic analysis on Lie groups , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[138]  Roberto Cipolla,et al.  Affine integral invariants for extracting symmetry axes , 1997, Image Vis. Comput..

[139]  Anuj Srivastava,et al.  Riemannian Analysis of Probability Density Functions with Applications in Vision , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[140]  David Mumford,et al.  Conformal shape representation , 2008 .

[141]  Michael I. Miller,et al.  Landmark matching via large deformation diffeomorphisms , 2000, IEEE Trans. Image Process..

[142]  Laurent Younes,et al.  Computable Elastic Distances Between Shapes , 1998, SIAM J. Appl. Math..

[143]  Anuj Srivastava,et al.  Removing Shape-Preserving Transformations in Square-Root Elastic (SRE) Framework for Shape Analysis of Curves , 2007, EMMCVPR.

[144]  Haibin Ling,et al.  Using the inner-distance for classification of articulated shapes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[145]  Daniel Cremers,et al.  Shape Matching by Variational Computation of Geodesics on a Manifold , 2006, DAGM-Symposium.

[146]  Fillia Makedon,et al.  Functional analysis of cardiac MR images using SPHARM modeling , 2005, SPIE Medical Imaging.

[147]  Ulf Grenander,et al.  On the Shape of Plane Images , 1993, SIAM J. Appl. Math..

[148]  A. Elgammal,et al.  Inferring 3D body pose from silhouettes using activity manifold learning , 2004, CVPR 2004.

[149]  T. K. Carne,et al.  Shape and Shape Theory , 1999 .

[150]  Alain Trouvé,et al.  Geodesic Shooting for Computational Anatomy , 2006, Journal of Mathematical Imaging and Vision.

[151]  Stephen R. Marsland,et al.  A minimum description length objective function for groupwise non-rigid image registration , 2008, Image Vis. Comput..

[152]  D. Kendall,et al.  The Riemannian Structure of Euclidean Shape Spaces: A Novel Environment for Statistics , 1993 .

[153]  I. Dryden,et al.  Shape-space smoothing splines for planar landmark data , 2007 .

[154]  Ivo D. Dinov,et al.  A Computational Model of Multidimensional Shape , 2010, International Journal of Computer Vision.