Moment Constraints in Convex Optimization for Segmentation and Tracking

Convex relaxation techniques have become a popular approach to shape optimization as they allow to compute solutions independent of initialization to a variety of problems. In this chapter, we will show that shape priors in terms of moment constraints can be imposed within the convex optimization framework, since they give rise to convex constraints. In particular, the lower-order moments correspond to the overall area, the centroid, and the variance or covariance of the shape and can be easily imposed in interactive segmentation methods. Respective constraints can be imposed as hard constraints or soft constraints. Quantitative segmentation studies on a variety of images demonstrate that the user can impose such constraints with a few mouse clicks, leading to substantial improvements of the resulting segmentation, and reducing the average segmentation error from 12 % to 0.35 %. GPU-based computation times of around 1 second allow for interactive segmentation.

[1]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[2]  S. Osher,et al.  Algorithms Based on Hamilton-Jacobi Formulations , 1988 .

[3]  Olga Veksler,et al.  Star Shape Prior for Graph-Cut Image Segmentation , 2008, ECCV.

[4]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[5]  R. Dykstra,et al.  A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces , 1986 .

[6]  Daniel Cremers,et al.  Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation , 2006, International Journal of Computer Vision.

[7]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[8]  Toby Sharp,et al.  Image segmentation with a bounding box prior , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[10]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[11]  Fabrice Heitz,et al.  Affine-invariant geometric shape priors for region-based active contours , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[13]  Daniel Cremers,et al.  Continuous Global Optimization in Multiview 3D Reconstruction , 2007, International Journal of Computer Vision.

[14]  Daniel Cremers,et al.  TVSeg - Interactive Total Variation Based Image Segmentation , 2008, BMVC.

[15]  Daniel Cremers,et al.  Integration of Multiview Stereo and Silhouettes Via Convex Functionals on Convex Domains , 2008, ECCV.

[16]  Ulf Grenander,et al.  Hands: A Pattern Theoretic Study of Biological Shapes , 1990 .

[17]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[18]  Daniel Cremers,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 a Combinatorial Solution for Model-based Image Segmentation and Real-time Tracking , 2022 .

[19]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[20]  Dima Damen,et al.  Detecting Carried Objects in Short Video Sequences , 2008, ECCV.

[21]  Anthony J. Yezzi,et al.  Gradient flows and geometric active contour models , 1995, Proceedings of IEEE International Conference on Computer Vision.

[22]  Daniel Cremers,et al.  A convex framework for image segmentation with moment constraints , 2011, 2011 International Conference on Computer Vision.

[23]  Renaud Keriven,et al.  Shape Priors using Manifold Learning Techniques , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[24]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[25]  Dima Damen,et al.  British Machine Vision Conference (BMVC) , 2007 .

[26]  E. Ising.  Beitrag zur Theorie des Ferromagnetismus , 1925 .

[27]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[28]  Martin S. Kochmanski.  NOTE ON THE E. ISING'S PAPER ,,BEITRAG ZUR THEORIE DES FERROMAGNETISMUS" (Zs. Physik, 31, 253 (1925)) , 2008 .

[29]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[30]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[31]  Olga Veksler,et al.  Semiautomatic segmentation with compact shape prior , 2009, Image Vis. Comput..