Spatially Varying Color Distributions for Interactive Multilabel Segmentation

We propose a method for interactive multilabel segmentation which explicitly takes into account the spatial variation of color distributions. To this end, we estimate a joint distribution over color and spatial location using a generalized Parzen density estimator applied to each user scribble. In this way, we obtain a likelihood for observing certain color values at a spatial coordinate. This likelihood is then incorporated in a Bayesian MAP estimation approach to multiregion segmentation which in turn is optimized using recently developed convex relaxation techniques. These guarantee global optimality for the two-region case (foreground/background) and solutions of bounded optimality for the multiregion case. We show results on the GrabCut benchmark, the recently published Graz benchmark, and on the Berkeley segmentation database which exceed previous approaches such as GrabCut [32], the Random Walker [15], Santner's approach [35], TV-Seg [39], and interactive graph cuts [4] in accuracy. Our results demonstrate that taking into account the spatial variation of color models leads to drastic improvements for interactive image segmentation.

[1]  Antonin Chambolle,et al.  Diagonal preconditioning for first order primal-dual algorithms in convex optimization , 2011, 2011 International Conference on Computer Vision.

[2]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  A. Fiacco.  A Finite Algorithm for Finding the Projection of a Point onto the Canonical Simplex of R " , 2009 .

[4]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Christoph Schnörr,et al.  Convex optimization for multi-class image labeling with a novel family of total variation based regularizers , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[6]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[8]  Daniel Cremers,et al.  A convex relaxation approach for computing minimal partitions , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Carsten Rother,et al.  Improving Color Modeling for Alpha Matting , 2008, BMVC.

[11]  R. B. Potts.  Some generalized order-disorder transformations , 1952, Mathematical Proceedings of the Cambridge Philosophical Society.

[12]  Daniel Cremers,et al.  Space-Varying Color Distributions for Interactive Multiregion Segmentation: Discrete versus Continuous Approaches , 2011, EMMCVPR.

[13]  Daniel Cremers,et al.  TVSeg - Interactive Total Variation Based Image Segmentation , 2008, BMVC.

[14]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[15]  Daniel Cremers,et al.  On Local Region Models and the Statistical Interpretation of the Piecewise Smooth Mumford-shah Functional , 2007 .

[16]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[17]  C. Michelot.  A finite algorithm for finding the projection of a point onto the canonical simplex of ∝n , 1986 .

[18]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[19]  Jian Sun,et al.  A global sampling method for alpha matting , 2011, CVPR 2011.

[20]  Daniel Cremers,et al.  Interactive Motion Segmentation , 2010, DAGM-Symposium.

[21]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[22]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[23]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[24]  Olivier Duchenne,et al.  Fast interactive segmentation using color and textural information Segmentation interactive basée sur la couleur et la texture , 2006 .

[25]  Horst Bischof,et al.  Interactive Multi-label Segmentation , 2010, ACCV.

[26]  T. O’Neil.  Geometric Measure Theory , 2002 .

[27]  Daniel Cremers,et al.  A convex approach for computing minimal partitions , 2008 .

[28]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[29]  Jan-Michael Frahm,et al.  Fast Global Labeling for Real-Time Stereo Using Multiple Plane Sweeps , 2008, VMV.

[30]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Patrick Pérez,et al.  Interactive Image Segmentation Using an Adaptive GMMRF Model , 2004, ECCV.

[32]  Pushmeet Kohli,et al.  Graph Cut Based Inference with Co-occurrence Statistics , 2010, ECCV.

[33]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[34]  H. Akaike.  An approximation to the density function , 1954 .

[35]  Chi-Keung Tang,et al.  Soft Color Segmentation and Its Applications , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Rachid Deriche,et al.  A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape , 2007, International Journal of Computer Vision.

[37]  Noel E. O'Connor,et al.  A comparative evaluation of interactive segmentation algorithms , 2010, Pattern Recognit..

[38]  Jing Yuan,et al.  Convex Multi-class Image Labeling by Simplex-Constrained Total Variation , 2009, SSVM.

[39]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[40]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[41]  Manuel Menezes de Oliveira Neto,et al.  Shared Sampling for Real‐Time Alpha Matting , 2010, Comput. Graph. Forum.