Semi-supervised Multiclass Clustering Based on Signed Total Variation

We consider the problem of semi-supervised clustering for multiple (more than two) classes. The proposed clustering algorithm uses the (dis)similarity of given data to learn the unknown cluster labels. We quantify label (dis)similarity in terms of the new concept of signed total variation (TV). The clustering task is formulated as a convex optimization problem with an ℓ1-norm regularization term that helps when only few labels are known. We solve the optimization problem by developing an ADMM-based algorithm whose per-iteration complexity scales linearly with the number of edges and the number of clusters. Our algorithm admits a distributed implementation and can therefore efficiently handle large-dimensional problems. Numerical experiments demonstrate the superiority of our scheme.

[1]  Stephen J. Wright,et al.  Dissimilarity in Graph-Based Semi-Supervised Classification , 2007, AISTATS.

[2]  Yuji Matsumoto,et al.  Using the Mutual k-Nearest Neighbor Graphs for Semi-supervised Classification on Natural Language Data , 2011, CoNLL.

[3]  Xavier Bresson,et al.  Multiclass Total Variation Clustering , 2013, NIPS.

[4]  A. Fiacco.  A Finite Algorithm for Finding the Projection of a Point onto the Canonical Simplex of R " , 2009 .

[5]  Konstantin Avrachenkov,et al.  Semi-supervised learning with regularized Laplacian , 2015, Optim. Methods Softw..

[6]  Mikhail Belkin,et al.  Beyond the point cloud: from transductive to semi-supervised learning , 2005, ICML.

[7]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[8]  Ian Davidson,et al.  Flexible constrained spectral clustering , 2010, KDD.

[9]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[10]  Antonin Chambolle,et al.  Diagonal preconditioning for first order primal-dual algorithms in convex optimization , 2011, 2011 International Conference on Computer Vision.

[11]  Yunzhang Zhu.  An Augmented ADMM Algorithm With Application to the Generalized Lasso Problem , 2017 .

[12]  Sahin Albayrak,et al.  Spectral Analysis of Signed Graphs for Clustering, Prediction and Visualization , 2010, SDM.

[13]  Xue-Cheng Tai,et al.  An Effective Region Force for Some Variational Models for Learning and Clustering , 2017, Journal of Scientific Computing.

[14]  Ian Davidson,et al.  On constrained spectral clustering and its applications , 2012, Data Mining and Knowledge Discovery.

[15]  Gerald Matz,et al.  SEMI-SUPERVISED CLUSTERING BASED ON SIGNED TOTAL VARIATION , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[16]  Avrim Blum,et al.  Learning from Labeled and Unlabeled Data using Graph Mincuts , 2001, ICML.

[17]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[18]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[19]  Matthias Hein,et al.  Tight Continuous Relaxation of the Balanced k-Cut Problem , 2014, NIPS.

[20]  Dit-Yan Yeung,et al.  Robust path-based spectral clustering , 2008, Pattern Recognit..

[21]  Gerald Matz,et al.  Graph Signal Recovery via Primal-Dual Algorithms for Total Variation Minimization , 2017, IEEE Journal of Selected Topics in Signal Processing.

[22]  Wei Liu,et al.  Robust and Scalable Graph-Based Semisupervised Learning , 2012, Proceedings of the IEEE.

[23]  Miguel Á. Carreira-Perpiñán,et al.  Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application , 2013, ArXiv.

[24]  C. Michelot.  A finite algorithm for finding the projection of a point onto the canonical simplex of ∝n , 1986 .