Global Binary Optimization on Graphs for Classification of High-Dimensional Data

This work develops a global minimization framework for segmentation of high-dimensional data into two classes. It combines recent convex optimization methods from imaging with recent graph- based variational models for data segmentation. Two convex splitting algorithms are proposed, where graph-based PDE techniques are used to solve some of the subproblems. It is shown that global minimizers can be guaranteed for semi-supervised segmentation with two regions. If constraints on the volume of the regions are incorporated, global minimizers cannot be guaranteed, but can often be obtained in practice and otherwise be closely approximated. Experiments on benchmark data sets show that our models produce segmentation results that are comparable with or outperform the state-of-the-art algorithms. In particular, we perform a thorough comparison to recent MBO (Merriman–Bence–Osher, AMS-Selected Lectures in Mathematics Series: Computational Crystal Growers Workshop, 1992) and phase field methods, and show the advantage of the algorithms proposed in this paper.

[1]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[2]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[3]  Steven J. Ruuth Efficient Algorithms for Diffusion-Generated Motion by Mean Curvature , 1998 .

[4]  Bernhard Schölkopf,et al.  Training Invariant Support Vector Machines , 2002, Machine Learning.

[5]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Xavier Bresson,et al.  Multiclass Total Variation Clustering , 2013, NIPS.

[7]  Eric V. Denardo,et al.  Flows in Networks , 2011 .

[8]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Xiaojin Zhu,et al.  Semi-Supervised Learning Literature Survey , 2005 .

[10]  Xavier Bresson,et al.  An Adaptive Total Variation Algorithm for Computing the Balanced Cut of a Graph , 2013, 1302.2717.

[11]  A. Bertozzi,et al.  $\Gamma$-convergence of graph Ginzburg-Landau functionals , 2012, Advances in Differential Equations.

[12]  A. Chambolle Practical, Unified, Motion and Missing Data Treatment in Degraded Video , 2004, Journal of Mathematical Imaging and Vision.

[13]  Daniel Cremers,et al.  A convex framework for image segmentation with moment constraints , 2011, 2011 International Conference on Computer Vision.

[14]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[15]  Xavier Bresson,et al.  Multi-class Transductive Learning Based on ℓ1 Relaxations of Cheeger Cut and Mumford-Shah-Potts Model , 2013, Journal of Mathematical Imaging and Vision.

[16]  Leo Grady,et al.  Multilabel random walker image segmentation using prior models , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  T. Chan,et al.  Primal dual algorithms for convex models and applications to image restoration, registration and nonlocal inpainting , 2010 .

[18]  B. Mohar THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[19]  Arjuna Flenner,et al.  Diffuse Interface Models on Graphs for Classification of High Dimensional Data , 2012, SIAM Rev..

[20]  Shih-Fu Chang,et al.  Graph transduction via alternating minimization , 2008, ICML '08.

[21]  BressonXavier,et al.  Geometric Applications of the Split Bregman Method , 2010 .

[22]  Luca Maria Gambardella,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Flexible, High Performance Convolutional Neural Networks for Image Classification , 2022 .

[23]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[24]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[26]  Xavier Bresson,et al.  Convergence and Energy Landscape for Cheeger Cut Clustering , 2012, NIPS.

[27]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[28]  Dani Lischinski,et al.  Spectral Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Balázs Kégl,et al.  Boosting products of base classifiers , 2009, ICML '09.

[30]  Arthur D. Szlam,et al.  Total variation and cheeger cuts , 2010, ICML 2010.

[31]  Arthur D. Szlam,et al.  A Total Variation-based Graph Clustering Algorithm for Cheeger Ratio Cuts , 2009 .

[32]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[33]  Xue-Cheng Tai,et al.  A spatially continuous max-flow and min-cut framework for binary labeling problems , 2014, Numerische Mathematik.

[34]  Simon Setzer,et al.  Operator Splittings, Bregman Methods and Frame Shrinkage in Image Processing , 2011, International Journal of Computer Vision.

[35]  Xavier Bresson,et al.  Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction , 2010, J. Sci. Comput..

[36]  Bernhard Schölkopf,et al.  Semi-Supervised Learning (Adaptive Computation and Machine Learning) , 2006 .

[37]  Camille Couprie,et al.  Power Watershed: A Unifying Graph-Based Optimization Framework , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  B. Schölkopf,et al.  A Regularization Framework for Learning from Graph Data , 2004, ICML 2004.

[39]  Xavier Bresson,et al.  Fast Global Minimization of the Active Contour/Snake Model , 2007, Journal of Mathematical Imaging and Vision.

[40]  Daniel Cremers,et al.  Efficient Convex Optimization for Minimal Partition Problems with Volume Constraints , 2013, EMMCVPR.

[41]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Xue-Cheng Tai,et al.  A study on continuous max-flow and min-cut approaches , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Xue-Cheng Tai,et al.  Augmented Lagrangian Method, Dual Methods, and Split Bregman Iteration for ROF, Vectorial TV, and High Order Models , 2010, SIAM J. Imaging Sci..

[45]  J. Cheeger A lower bound for the smallest eigenvalue of the Laplacian , 1969 .

[46]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[47]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[48]  Mason A. Porter,et al.  A Method Based on Total Variation for Network Modularity Optimization Using the MBO Scheme , 2013, SIAM J. Appl. Math..

[49]  Michael William Newman,et al.  The Laplacian spectrum of graphs , 2001 .

[50]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[51]  A. Bertozzi,et al.  Mean Curvature, Threshold Dynamics, and Phase Field Theory on Finite Graphs , 2013, 1307.0045.

[52]  Rüdiger Westermann,et al.  RANDOM WALKS FOR INTERACTIVE ALPHA-MATTING , 2005 .

[53]  Christopher R. Anderson,et al.  A Rayleigh-Chebyshev procedure for finding the smallest eigenvalues and associated eigenvectors of large sparse Hermitian matrices , 2010, J. Comput. Phys..

[54]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[55]  Xue-Cheng Tai,et al.  Image Segmentation Using Some Piecewise Constant Level Set Methods with MBO Type of Projection , 2007, International Journal of Computer Vision.

[56]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[57]  Camille Couprie,et al.  Combinatorial Continuous Maximum Flow , 2010, SIAM J. Imaging Sci..

[58]  T. Chan NON-LOCAL UNSUPERVISED VARIATIONAL IMAGE SEGMENTATION MODELS , 2008 .

[59]  Matthias Hein,et al.  An Inverse Power Method for Nonlinear Eigenproblems with Applications in 1-Spectral Clustering and Sparse PCA , 2010, NIPS.

[60]  Andrea L. Bertozzi,et al.  An MBO Scheme on Graphs for Classification and Image Processing , 2013, SIAM J. Imaging Sci..

[61]  Ronald R. Coifman,et al.  Regularization on Graphs with Function-adapted Diffusion Processes , 2008, J. Mach. Learn. Res..

[62]  Xavier Bresson,et al.  Total Variation, Cheeger Cuts , 2010, ICML.

[63]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[64]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[65]  Leo Grady,et al.  Discrete Calculus - Applied Analysis on Graphs for Computational Science , 2010 .

[66]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[67]  Abderrahim Elmoataz,et al.  Nonlocal PdES on graphs for active contours models with applications to image segmentation and data clustering , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[68]  Abderrahim Elmoataz,et al.  Nonlocal Discrete Regularization on Weighted Graphs: A Framework for Image and Manifold Processing , 2008, IEEE Transactions on Image Processing.

[69]  Camille Couprie,et al.  Combinatorial Continuous Maximal Flows , 2010, ArXiv.

[70]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.