Globally Optimal Joint Image Segmentation and Shape Matching Based on Wasserstein Modes

A functional for joint variational object segmentation and shape matching is developed. The formulation is based on optimal transport w.r.t. geometric distance and local feature similarity. Geometric invariance and modelling of object-typical statistical variations is achieved by introducing degrees of freedom that describe transformations and deformations of the shape template. The shape model is mathematically equivalent to contour-based approaches but inference can be performed without conversion between the contour and region representations, allowing combination with other convex segmentation approaches and simplifying optimization. While the overall functional is non-convex, non-convexity is confined to a low-dimensional variable. We propose a locally optimal alternating optimization scheme and a globally optimal branch and bound scheme, based on adaptive convex relaxation. Combining both methods allows to eliminate the delicate initialization problem inherent to many contour based approaches while remaining computationally practical. The properties of the functional, its ability to adapt to a wide range of input data structures and the different optimization schemes are illustrated and compared by numerical experiments.

[1]  Alain Trouvé,et al.  Functional Currents: A New Mathematical Tool to Model and Analyse Functional Shapes , 2012, Journal of Mathematical Imaging and Vision.

[2]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Lei Zhu,et al.  Optimal Mass Transport for Registration and Warping , 2004, International Journal of Computer Vision.

[4]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[5]  J. Lott Some Geometric Calculations on Wasserstein Space , 2006, math/0612562.

[6]  Arnaud Doucet,et al.  Fast Computation of Wasserstein Barycenters , 2013, ICML.

[7]  Martin Rumpf,et al.  Time‐Discrete Geodesics in the Space of Shells , 2012, Comput. Graph. Forum.

[8]  Andrew Zisserman,et al.  OBJ CUT , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Christoph Schnörr,et al.  Continuous Multiclass Labeling Approaches and Algorithms , 2011, SIAM J. Imaging Sci..

[10]  L. Ambrosio,et al.  A User’s Guide to Optimal Transport , 2013 .

[11]  Benjamin Berkels,et al.  Discrete Geodesic Regression in Shape Space , 2013, EMMCVPR.

[12]  Andrew Blake,et al.  Image Segmentation by Branch-and-Mincut , 2008, ECCV.

[13]  C. Villani Optimal Transport: Old and New , 2008 .

[14]  Alain Trouvé,et al.  Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms , 2005, International Journal of Computer Vision.

[15]  Christoph Schnörr,et al.  Object Segmentation by Shape Matching with Wasserstein Modes , 2013, EMMCVPR.

[16]  Olivier D. Faugeras,et al.  Approximations of Shape Metrics and Application to Shape Warping and Empirical Shape Statistics , 2005, Found. Comput. Math..

[17]  Stefano Soatto,et al.  A New Geometric Metric in the Space of Curves, and Applications to Tracking Deforming Objects by Prediction and Filtering , 2011, SIAM J. Imaging Sci..

[18]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[19]  Leonidas J. Guibas,et al.  The Earth Mover's Distance under transformation sets , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[20]  Alain Trouvé,et al.  Diffeomorphic matching of distributions: a new approach for unlabelled point-sets and sub-manifolds matching , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[21]  Guillaume Carlier,et al.  Barycenters in the Wasserstein Space , 2011, SIAM J. Math. Anal..

[22]  Dan Roth,et al.  Learning to detect objects in images via a sparse, part-based representation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Christoph Schnörr,et al.  Modelling Convex Shape Priors and Matching Based on the Gromov-Wasserstein Distance , 2013, Journal of Mathematical Imaging and Vision.

[24]  Dmitry Vetrov,et al.  Learning a Model for Shape-Constrained Image Segmentation from Weakly Labeled Data , 2013, EMMCVPR.

[25]  D. Mumford,et al.  A Metric on Shape Space with Explicit Geodesics , 2007, 0706.4299.

[26]  Christoph Schnörr,et al.  Contour Manifolds and Optimal Transport , 2013, ArXiv.

[27]  L. Younes Shapes and Diffeomorphisms , 2010 .

[28]  Daniel Cremers,et al.  Shape statistics in kernel space for variational image segmentation , 2003, Pattern Recognit..

[29]  Facundo Mémoli,et al.  Gromov–Wasserstein Distances and the Metric Approach to Object Matching , 2011, Found. Comput. Math..

[30]  Dirk Helbing,et al.  Modelling and Optimisation of Flows on Networks , 2013 .

[31]  L. Younes,et al.  Diffeomorphic matching of distributions: a new approach for unlabelled point-sets and sub-manifolds matching , 2004, CVPR 2004.

[32]  Daniel Cremers,et al.  A convex relaxation approach for computing minimal partitions , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[34]  D. Mumford,et al.  Riemannian Geometries on Spaces of Plane Curves , 2003, math/0312384.

[35]  Daniel Cremers,et al.  A convex framework for image segmentation with moment constraints , 2011, 2011 International Conference on Computer Vision.

[36]  Cordelia Schmid,et al.  Proceedings. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition , 2005 .

[37]  Alexander M. Bronstein,et al.  Efficient Computation of Isometry-Invariant Distances Between Surfaces , 2006, SIAM J. Sci. Comput..

[38]  Anuj Srivastava,et al.  On Shape of Plane Elastic Curves , 2007, International Journal of Computer Vision.

[39]  Michael Werman,et al.  Fast and robust Earth Mover's Distances , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  Z. Lähner Time-Discrete Geodesics in the Space of Shells Time-Discrete Geodesics in the Space of Shells , 2014 .

[41]  Gustavo K. Rohde,et al.  A Linear Optimal Transportation Framework for Quantifying and Visualizing Variations in Sets of Images , 2012, International Journal of Computer Vision.

[42]  P. Michor,et al.  The Convenient Setting of Global Analysis , 1997 .

[43]  HakerSteven,et al.  Optimal Mass Transport for Registration and Warping , 2004 .

[44]  Christoph Schnörr,et al.  Weakly Convex Coupling Continuous Cuts and Shape Priors , 2011, SSVM.