Higher-order segmentation via multicuts

We propose a novel and general formulation for hyper-graph correlation clustering.Any permutation invariant function can be included into a multicut problem.We provide a comparison of LP and ILP cutting plane methods and rounding procedures for the multicut problem.Many sparse Potts models can be solved to global optimality very efficient by the proposed method.The C++ implementations used in this manuscript is freely available online. Multicuts enable to conveniently represent discrete graphical models for unsupervised and supervised image segmentation, in the case of local energy functions that exhibit symmetries. The basic Potts model and natural extensions thereof to higher-order models provide a prominent class of such objectives, that cover a broad range of segmentation problems relevant to image analysis and computer vision. We exhibit a way to systematically take into account such higher-order terms for computational inference. Furthermore, we present results of a comprehensive and competitive numerical evaluation of a variety of dedicated cutting-plane algorithms. Our approach enables the globally optimal evaluation of a significant subset of these models, without compromising runtime. Polynomially solvable relaxations are studied as well, along with advanced rounding schemes for post-processing.

[1]  Gerhard Reinelt,et al.  Globally Optimal Image Partitioning by Multicuts , 2011, EMMCVPR.

[2]  Pushmeet Kohli,et al.  P³ & Beyond: Move Making Algorithms for Solving Higher Order Functions , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[4]  Mihalis Yannakakis,et al.  The Complexity of Multiterminal Cuts , 1994, SIAM J. Comput..

[5]  Endre Boros,et al.  Pseudo-Boolean optimization , 2002, Discret. Appl. Math..

[6]  Ullrich Köthe,et al.  The Lazy Flipper: Efficient Depth-Limited Exhaustive Search in Discrete Graphical Models , 2012, ECCV.

[7]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[8]  Ullrich Köthe,et al.  Probabilistic image segmentation with closedness constraints , 2011, 2011 International Conference on Computer Vision.

[9]  Amos Fiat,et al.  Correlation clustering in general weighted graphs , 2006, Theor. Comput. Sci..

[10]  Vladimir Kolmogorov,et al.  Computing geodesics and minimal surfaces via graph cuts , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  Richard S. Zemel,et al.  HOP-MAP: Efficient Message Passing with High Order Potentials , 2010, AISTATS.

[12]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[13]  Pushmeet Kohli,et al.  Minimizing sparse higher order energy functions of discrete variables , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  M. R. Rao,et al.  On the multiway cut polyhedron , 1991, Networks.

[15]  Martin Grötschel,et al.  Complete Descriptions of Small Multicut Polytopes , 1990, Applied Geometry And Discrete Mathematics.

[16]  Tommi S. Jaakkola,et al.  New Outer Bounds on the Marginal Polytope , 2007, NIPS.

[17]  Andrew Blake,et al.  Fusion Moves for Markov Random Field Optimization , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Daniel Cremers,et al.  A Survey and Comparison of Discrete and Continuous Multi-label Optimization Approaches for the Potts Model , 2013, International Journal of Computer Vision.

[19]  M. R. Rao,et al.  The partition problem , 1993, Math. Program..

[20]  Gruia Calinescu.  Multiway Cut , 2016, Encyclopedia of Algorithms.

[21]  Yuval Rabani,et al.  An improved approximation algorithm for multiway cut , 1998, STOC '98.

[22]  Ulrik Brandes,et al.  On Modularity Clustering , 2008, IEEE Transactions on Knowledge and Data Engineering.

[23]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[24]  Nikos Komodakis,et al.  Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey , 2013, Comput. Vis. Image Underst..

[25]  Hiroshi Ishikawa,et al.  Higher-order clique reduction in binary graph cut , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  David Sontag,et al.  Efficiently Searching for Frustrated Cycles in MAP Inference , 2012, UAI.

[27]  Tommi S. Jaakkola,et al.  Tightening LP Relaxations for MAP using Message Passing , 2008, UAI.

[28]  Daniel Cremers,et al.  A Survey and Comparison of Discrete and Continuous Multilabel Segmentation Approaches , 2013 .

[29]  Yi Li,et al.  Orientation Robust Text Line Detection in Natural Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Yoshiko Wakabayashi,et al.  A cutting plane algorithm for a clustering problem , 1989, Math. Program..

[31]  Nicole Immorlica,et al.  Efficient location area planning for personal communication systems , 2003, IEEE/ACM Transactions on Networking.

[32]  Sebastian Nowozin,et al.  A Comparative Study of Modern Inference Techniques for Discrete Energy Minimization Problems , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Sebastian Nowozin,et al.  Solution stability in linear programming relaxations: graph partitioning and unsupervised learning , 2009, ICML '09.

[34]  Camille Roth,et al.  Natural Scales in Geographical Patterns , 2017, Scientific Reports.

[35]  Andrew Zisserman,et al.  Pylon Model for Semantic Segmentation , 2011, NIPS.

[36]  Nikos Komodakis,et al.  Approximate Labeling via Graph Cuts Based on Linear Programming , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  P. Gritzmann,et al.  Applied geometry and discrete mathematics : the Victor Klee festschrift , 1991 .

[38]  Jacob Goldberger,et al.  Ensemble Segmentation Using Efficient Integer Linear Programming , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Ullrich Köthe,et al.  Globally Optimal Closed-Surface Segmentation for Connectomics , 2012, ECCV.

[40]  Pushmeet Kohli,et al.  Robust Higher Order Potentials for Enforcing Label Consistency , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Nikos Komodakis,et al.  Beyond pairwise energies: Efficient optimization for higher-order MRFs , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Dániel Marx,et al.  A Tight Lower Bound for Planar Multiway Cut with Fixed Number of Terminals , 2012, ICALP.

[43]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[44]  Michel Deza,et al.  Geometry of cuts and metrics , 2009, Algorithms and combinatorics.

[45]  Gerhard Reinelt,et al.  Towards Efficient and Exact MAP-Inference for Large Scale Discrete Computer Vision Problems via Combinatorial Optimization , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Yuri Boykov,et al.  Globally optimal segmentation of multi-region objects , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[47]  Michael J. Black,et al.  Efficient Belief Propagation with Learned Higher-Order Markov Random Fields , 2006, ECCV.

[48]  Sebastian Nowozin,et al.  A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems , 2014, International Journal of Computer Vision.

[49]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[50]  Nicole Immorlica,et al.  Efficient location area planning for personal communication systems , 2006, TNET.

[51]  Chen Wang,et al.  A Primal-Dual Algorithm for Higher-Order Multilabel Markov Random Fields , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Sebastian Nowozin,et al.  Higher-Order Correlation Clustering for Image Segmentation , 2011, NIPS.

[53]  Mihalis Yannakakis,et al.  The complexity of multiway cuts (extended abstract) , 1992, STOC '92.

[54]  Anton Osokin,et al.  Fast Approximate Energy Minimization with Label Costs , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[55]  M. Meilă.  Comparing clusterings---an information based distance , 2007 .

[56]  N. Metropolis,et al.  An Efficient Heuristic Procedure for Partitioning Graphs , 2017 .

[57]  W. Marsden.  I and J , 2012 .

[58]  Pushmeet Kohli,et al.  Dynamic Hybrid Algorithms for MAP Inference in Discrete MRFs , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  PotetzBrian,et al.  Efficient belief propagation for higher-order cliques using linear constraint nodes , 2008 .

[60]  Hongdong Li,et al.  Connected contours: A new contour completion model that respects the closure effect , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Martin Grötschel,et al.  Clique-Web Facets for Multicut Polytopes , 1992, Math. Oper. Res..

[62]  Pushmeet Kohli,et al.  Inference Methods for CRFs with Co-occurrence Statistics , 2012, International Journal of Computer Vision.

[63]  M. Pawan Kumar,et al.  Parsimonious Labeling , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[64]  Olga Veksler,et al.  Markov random fields with efficient approximations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[65]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Endre Boros,et al.  A graph cut algorithm for higher-order Markov Random Fields , 2011, 2011 International Conference on Computer Vision.

[67]  Vladimir Kolmogorov,et al.  Submodular decomposition framework for inference in associative Markov networks with global constraints , 2011, CVPR 2011.

[68]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[69]  Fred W. Glover,et al.  Technical Note - Converting the 0-1 Polynomial Programming Problem to a 0-1 Linear Program , 1974, Oper. Res..

[70]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[71]  Éva Tardos,et al.  Approximation algorithms for classification problems with pairwise relationships: metric labeling and Markov random fields , 2002, JACM.

[72]  Julian Yarkony,et al.  Fast Planar Correlation Clustering for Image Segmentation , 2012, ECCV.

[73]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[74]  Tai Sing Lee,et al.  Efficient belief propagation for higher-order cliques using linear constraint nodes , 2008, Comput. Vis. Image Underst..

[75]  Sebastian Nowozin,et al.  Task-Specific Image Partitioning , 2013, IEEE Transactions on Image Processing.

[76]  Brian W. Kernighan,et al.  An efficient heuristic procedure for partitioning graphs , 1970, Bell Syst. Tech. J..

[77]  Sebastian Nowozin,et al.  Learning with structured data: applications to computer vision , 2009 .

[78]  Michael I. Jordan.  Graphical Models , 1998 .

[79]  H KappesJörg,et al.  A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems , 2015 .