Inexact Block Coordinate Descent Methods for Symmetric Nonnegative Matrix Factorization

Symmetric nonnegative matrix factorization (SNMF) is equivalent to computing a symmetric nonnegative low rank approximation of a data similarity matrix. It inherits the good data interpretability of the well-known nonnegative matrix factorization technique and has better ability of clustering nonlinearly separable data. In this paper, we focus on the algorithmic aspect of the SNMF problem and propose simple inexact block coordinate decent methods to address the problem, leading to both serial and parallel algorithms. The proposed algorithms have guaranteed convergence to stationary solutions and can efficiently handle large-scale and/or sparse SNMF problems. Extensive simulations verify the effectiveness of the proposed algorithms compared to recent state-of-the-art algorithms.

[1]  Ioannis D. Schizas,et al.  Distributed Informative-Sensor Identification via Sparsity-Aware Matrix Decomposition , 2013, IEEE Transactions on Signal Processing.

[2]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[3]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[4]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[5]  Meisam Razaviyayn,et al.  Successive Convex Approximation: Analysis and Applications , 2014 .

[6]  Songtao Lu,et al.  A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality , 2017, IEEE Transactions on Signal Processing.

[7]  P. H. Diananda On non-negative forms in real variables some or all of which are non-negative , 1962, Mathematical Proceedings of the Cambridge Philosophical Society.

[8]  Inderjit S. Dhillon,et al.  Coordinate Descent Methods for Symmetric Nonnegative Matrix Factorization , 2015, ArXiv.

[9]  Haesun Park,et al.  SymNMF: nonnegative low-rank approximation of a similarity matrix for graph clustering , 2014, Journal of Global Optimization.

[10]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[11]  Inderjit S. Dhillon,et al.  Semi-supervised graph clustering: a kernel approach , 2005, Machine Learning.

[12]  P. Tseng Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[13]  Inderjit S. Dhillon,et al.  Efficient and Non-Convex Coordinate Descent for Symmetric Nonnegative Matrix Factorization , 2015, IEEE Transactions on Signal Processing.

[14]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[15]  Stephen J. Wright Coordinate descent algorithms , 2015, Mathematical Programming.

[16]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[17]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[18]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[19]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[20]  Songtao Lu,et al.  A nonconvex splitting method for symmetric nonnegative matrix factorization: Convergence analysis and optimality , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  Philip S. Yu,et al.  Relational clustering by symmetric convex coding , 2007, ICML '07.

[22]  Zhi-Quan Luo,et al.  Parallel Successive Convex Approximation for Nonsmooth Nonconvex Optimization , 2014, NIPS.

[23]  Zhaoshui He,et al.  Symmetric Nonnegative Matrix Factorization: Algorithms and Applications to Probabilistic Clustering , 2011, IEEE Transactions on Neural Networks.

[24]  Peter J. C. Dickinson,et al.  On the computational complexity of membership problems for the completely positive cone and its dual , 2014, Comput. Optim. Appl..

[25]  Chris H. Q. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering , 2005, SDM.

[26]  Philip S. Yu,et al.  Co-clustering by block value decomposition , 2005, KDD '05.

[27]  M. Hall,et al.  Copositive and completely positive quadratic forms , 1963, Mathematical Proceedings of the Cambridge Philosophical Society.

[28]  Donald E. Knuth,et al.  The art of computer programming. Vol.2: Seminumerical algorithms , 1981 .