A Survey and Comparison of Discrete and Continuous Multilabel Segmentation Approaches

We present a survey and a comparison of a variety of algorithms that have been proposed over the years to minimize multilabel optimization problems based on the Potts model. Discrete approaches based on Markov Random Fields (MRFs) as well as continuous optimization approaches based on partial differential equations (PDEs) can be applied to the task. In contrast to the case of binary labeling, the multilabel problem is known to be NP hard and thus one can only expect near-optimal solutions. In this paper, we carry out a theoretical comparison and an experimental analysis of existing approaches with respect to accuracy, optimality and runtime, aimed at bringing out the advantages and short-comings of the respective algorithms. Systematic quantitative comparison is done on the Graz interactive image segmentation benchmark. This paper thereby generalizes a previous experimental comparison [18] from the binary to the multilabel case.

[1]  Antonin Chambolle,et al.  Diagonal preconditioning for first order primal-dual algorithms in convex optimization , 2011, 2011 International Conference on Computer Vision.

[2]  Daniel Cremers,et al.  Generalized ordering constraints for multilabel optimization , 2011, 2011 International Conference on Computer Vision.

[3]  Olga Veksler.  Multi-label Moves for MRFs with Truncated Convex Priors , 2009, EMMCVPR.

[4]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[5]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[7]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  A. Fiacco.  A Finite Algorithm for Finding the Projection of a Point onto the Canonical Simplex of R " , 2009 .

[9]  Daniel Cremers,et al.  Anisotropic Minimal Surfaces Integrating Photoconsistency and Normal Information for Multiview Stereo , 2010, ECCV.

[10]  Andrew Blake,et al.  LogCut - Efficient Graph Cut Optimization for Markov Random Fields , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[11]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[12]  Daniel Cremers,et al.  Multiphase Dynamic Labeling for Variational Recognition-Driven Image Segmentation , 2004, ECCV.

[13]  Hiroshi Ishikawa,et al.  Exact Optimization for Markov Random Fields with Convex Priors , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Marc Pollefeys,et al.  What is optimized in tight convex relaxations for multi-label problems? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[16]  Daniel Cremers,et al.  An algorithm for minimizing the Mumford-Shah functional , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Nikos Komodakis,et al.  Fast, Approximately Optimal Solutions for Single and Dynamic MRFs , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[19]  Pushmeet Kohli,et al.  Dynamic Hybrid Algorithms for MAP Inference in Discrete MRFs , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Vladimir Kolmogorov,et al.  Submodular decomposition framework for inference in associative Markov networks with global constraints , 2011, CVPR 2011.

[21]  Christoph Schnörr,et al.  Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem , 2012, Journal of Mathematical Imaging and Vision.

[22]  John Staples,et al.  The Maximum Flow Problem is Log Space Complete for P , 1982, Theor. Comput. Sci..

[23]  Marc Niethammer,et al.  Continuous maximal flows and Wulff shapes: Application to MRFs , 2009, CVPR.

[24]  Rachid Deriche,et al.  A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape , 2007, International Journal of Computer Vision.

[25]  Olga Veksler,et al.  Tiered scene labeling with dynamic programming , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Nikos Komodakis,et al.  A new framework for approximate labeling via graph cuts , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[27]  W. Eric L. Grimson,et al.  Model-based curve evolution technique for image segmentation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[28]  Daniel Cremers,et al.  An Experimental Comparison of Discrete and Continuous Shape Optimization Methods , 2008, ECCV.

[29]  Daniel Cremers,et al.  Spatially Varying Color Distributions for Interactive Multilabel Segmentation , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Christoph Schnörr,et al.  Convex optimization for multi-class image labeling with a novel family of total variation based regularizers , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[31]  Daniel Cremers,et al.  A Multiphase Dynamic Labeling Model for Variational Recognition-driven Image Segmentation , 2005, International Journal of Computer Vision.

[32]  Pushmeet Kohli,et al.  Making the right moves: Guiding alpha-expansion using local primal-dual gaps , 2011, CVPR 2011.

[33]  Daniel Cremers,et al.  Global Solutions of Variational Models with Convex Regularization , 2010, SIAM J. Imaging Sci..

[34]  Olga Veksler.  Graph Cut Based Optimization for MRFs with Truncated Convex Priors , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Olga Veksler,et al.  Order-Preserving Moves for Graph-Cut-Based Optimization , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Horst Bischof,et al.  Interactive Multi-label Segmentation , 2010, ACCV.

[37]  Éva Tardos,et al.  Approximation algorithms for classification problems with pairwise relationships: metric labeling and Markov random fields , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[38]  Jan-Michael Frahm,et al.  Fast Global Labeling for Real-Time Stereo Using Multiple Plane Sweeps , 2008, VMV.

[39]  Jing Yuan,et al.  Convex Multi-class Image Labeling by Simplex-Constrained Total Variation , 2009, SSVM.

[40]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Daniel Cremers,et al.  A convex approach for computing minimal partitions , 2008 .

[42]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields , 2006, ECCV.

[43]  Daniel Cremers,et al.  Space-Varying Color Distributions for Interactive Multiregion Segmentation: Discrete versus Continuous Approaches , 2011, EMMCVPR.

[44]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[46]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .