Convergence of a Steepest Descent Algorithm for Ratio Cut Clustering

AbstractUnsupervised clustering of scattered, noisy and high-dimensional data points is an important anddifficult problem. Tight continuous relaxations of balanced cut problems have recently been shown toprovide excellent clustering results. In this paper, we present an explicit-implicit gradient flow schemefor the relaxed ratio cut problem, and prove that the algorithm converges to a critical point of the energy.We also show the efficiency of the proposed algorithm on the two moons dataset. 1 Introduction Partitioning data points into sensible groups is a fundamental problem in machine learning and has a widerange of applications. An efficient approach to deal with this problem is to cast the data partitioningproblem as a graph clustering problem. Given a set of data points V = {x 1 ,...,x n } and similarityweights {w i,j } 1≤i,j≤n , the clustering problem aims at finding a balanced cut of the graph of the data.In this work, we consider the balanced cut of Hagen and Kahng [5] known as ratio cut. The ratio cutproblem isMinimize RatioCut(S) =P

[1]  Arthur D. Szlam,et al.  Total variation and cheeger cuts , 2010, ICML 2010.

[2]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[3]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[4]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[5]  Xavier Bresson,et al.  Multi-class Transductive Learning Based on ℓ1 Relaxations of Cheeger Cut and Mumford-Shah-Potts Model , 2013, Journal of Mathematical Imaging and Vision.

[6]  Matthias Hein,et al.  Spectral clustering based on the graph p-Laplacian , 2009, ICML '09.

[7]  Matthias Hein,et al.  Beyond Spectral Clustering - Tight Relaxations of Balanced Graph Cuts , 2011, NIPS.

[8]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[9]  Robert R. Meyer,et al.  Sufficient Conditions for the Convergence of Monotonic Mathematical Programming Algorithms , 1976, J. Comput. Syst. Sci..

[10]  Matthias Hein,et al.  An Inverse Power Method for Nonlinear Eigenproblems with Applications in 1-Spectral Clustering and Sparse PCA , 2010, NIPS.

[11]  Matthias Hein,et al.  Constrained 1-Spectral Clustering , 2012, AISTATS.

[12]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[13]  A. Ostrowski Solution of equations in Euclidean and Banach spaces , 1973 .