A Fast Continuous Max-Flow Approach to Non-convex Multi-labeling Problems

This paper studies continuous image labeling problems with an arbitrary data term and a total variation regularizer, where the labels are constrained to a finite set of real numbers. Inspired by Ishikawa’s multi-layered graph construction for the same labeling problem over a discrete image domain, we propose a novel continuous max-flow model and build up its duality to a convex relaxed formulation of image labeling under a new variational perspective. Via such continuous max-flow formulations, we show that exact and global optimizers can be obtained to the original non-convex labeling problem. We also extend the studies to problems with continuous-valued labels and introduce a new theory to this problem. Finally, we show the proposed continuous max-flow models directly lead to new fast flow-maximization algorithmic schemes which outperform previous approaches in terms of efficiency. Such continuous max-flow based algorithms can be validated by convex optimization theories and accelerated by modern parallel computational hardware.

[1]  R. B. Potts Some generalized order-disorder transformations , 1952, Mathematical Proceedings of the Cambridge Philosophical Society.

[2]  D. R. Fulkerson,et al.  Maximal Flow Through a Network , 1956 .

[3]  D. R. Fulkerson,et al.  Flows in Networks. , 1964 .

[4]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[5]  E. Giusti Minimal surfaces and functions of bounded variation , 1977 .

[6]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Gilbert Strang,et al.  Maximal flow through a domain , 1983, Math. Program..

[8]  M. Giaquinta Cartesian currents in the calculus of variations , 1983 .

[9]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[10]  A. Goldberg,et al.  A new approach to the maximum-flow problem , 1988, JACM.

[11]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[12]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[13]  M. Shashkov,et al.  Adjoint operators for the natural discretizations of the divergence gradient and curl on logically rectangular grids , 1997 .

[14]  M. Shashkov,et al.  Natural discretizations for the divergence, gradient, and curl on logically rectangular grids☆ , 1997 .

[15]  G. D. Maso,et al.  The calibration method for the Mumford-Shah functional and free-discontinuity problems , 1999, math/0105013.

[16]  J. Sivaloganathan CARTESIAN CURRENTS IN THE CALCULUS OF VARIATIONS I: CARTESIAN CURRENTS (Ergebnisse der Mathematik und ihrer Grenzgebiete (3), A Series of Modern Surveys in Mathematics 37) , 2000 .

[17]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[19]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[20]  Endre Boros,et al.  Pseudo-Boolean optimization , 2002, Discret. Appl. Math..

[21]  Vladimir Kolmogorov,et al.  Computing geodesics and minimal surfaces via graph cuts , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  Hiroshi Ishikawa,et al.  Exact Optimization for Markov Random Fields with Convex Priors , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Rachid Deriche,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004 .

[24]  Tony F. Chan,et al.  A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model , 2002, International Journal of Computer Vision.

[25]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[27]  R. Zabih,et al.  What energy functions can be minimized via graph cuts , 2004 .

[28]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[30]  Vladimir Kolmogorov,et al.  What metrics can be approximated by geo-cuts, or global optimization of length/area and flux , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[31]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[32]  Xuecheng Tai,et al.  Piecewise Constant Level Set Methods for Multiphase Motion , 2005 .

[33]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[34]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[35]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[36]  Hugues Talbot,et al.  Globally minimal surfaces by continuous maximal flows , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Xue-Cheng Tai,et al.  A variant of the level set method and applications to image segmentation , 2006, Math. Comput..

[38]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Vladimir Kolmogorov,et al.  Minimizing Nonsubmodular Functions with Graph Cuts-A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Xavier Bresson,et al.  Fast Global Minimization of the Active Contour/Snake Model , 2007, Journal of Mathematical Imaging and Vision.

[41]  Victor S. Lempitsky,et al.  Global Optimization for Shape Fitting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Olga Veksler Graph Cut Based Optimization for MRFs with Truncated Convex Priors , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Yoon Mo Jung,et al.  Multiphase Image Segmentation via Modica-Mortola Phase Transition , 2007, SIAM J. Appl. Math..

[44]  Nikos Komodakis,et al.  Approximate Labeling via Graph Cuts Based on Linear Programming , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Andrea L. Bertozzi,et al.  Inpainting of Binary Images Using the Cahn–Hilliard Equation , 2007, IEEE Transactions on Image Processing.

[46]  Daniel Cremers,et al.  A Convex Formulation of Continuous Multi-label Problems , 2008, ECCV.

[47]  Jan-Michael Frahm,et al.  Fast Global Labeling for Real-Time Stereo Using Multiple Plane Sweeps , 2008, VMV.

[48]  Daniel Cremers,et al.  A convex approach for computing minimal partitions , 2008 .

[49]  Xue-Cheng Tai,et al.  Graph Cut Optimization for the Piecewise Constant Level Set Method Applied to Multiphase Image Segmentation , 2009, SSVM.

[50]  Pushmeet Kohli,et al.  P³ & Beyond: Move Making Algorithms for Solving Higher Order Functions , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Daniel Cremers,et al.  An algorithm for minimizing the Mumford-Shah functional , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[52]  Jérôme Darbon,et al.  Global optimization for first order Markov Random Fields with submodular priors , 2008, Discret. Appl. Math..

[53]  Xue-Cheng Tai,et al.  Efficient Global Minimization for the Multiphase Chan-Vese Model of Image Segmentation , 2009, EMMCVPR.

[54]  A. Chambolle,et al.  A convex relaxation approach for computing minimal partitions , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Daniel Cremers,et al.  A convex relaxation approach for computing minimal partitions , 2009, CVPR.

[56]  Hiroshi Ishikawa Higher-order clique reduction in binary graph cut , 2009, CVPR.

[57]  Jing Yuan,et al.  Convex Multi-class Image Labeling by Simplex-Constrained Total Variation , 2009, SSVM.

[58]  Xue-Cheng Tai,et al.  A Continuous Max-Flow Approach to Potts Model , 2010, ECCV.

[59]  Xue-Cheng Tai,et al.  Global Minimization for Continuous Multiphase Partitioning Problems Using a Dual Approach , 2011, International Journal of Computer Vision.

[60]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[61]  T. Chan,et al.  Primal dual algorithms for convex models and applications to image restoration, registration and nonlocal inpainting , 2010 .

[62]  Christoph Schnörr,et al.  Fast and Exact Primal-Dual Iterations for Variational Problems in Computer Vision , 2010, ECCV.

[63]  Daniel Cremers,et al.  Global Solutions of Variational Models with Convex Regularization , 2010, SIAM J. Imaging Sci..

[64]  Xue-Cheng Tai,et al.  A study on continuous max-flow and min-cut approaches , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[65]  S. Osher,et al.  Global minimization of Markov random fields with applications to optical flow , 2012 .