Coordinate Descent Methods for Symmetric Nonnegative Matrix Factorization

Given a symmetric nonnegative matrix A, symmetric nonnegative matrix factorization (symNMF) is the problem of finding a nonnegative matrix H , usually with much fewer columns than A, such that A ≈ HH . SymNMF can be used for data analysis and in particular for various clustering tasks. In this paper, we propose simple and very efficient coordinate descent schemes to solve this problem, and that can handle large and sparse input matrices. The effectiveness of our methods is illustrated on synthetic and real-world data sets, and we show that they perform favorably compared to recent state-of-the-art methods.

[1]  Efstratios Gallopoulos,et al.  Computing symmetric nonnegative rank factorizations , 2012 .

[2]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[3]  Peter J. C. Dickinson,et al.  On the computational complexity of membership problems for the completely positive cone and its dual , 2014, Comput. Optim. Appl..

[4]  Nicolas Gillis,et al.  Solving the Maximum Clique Problem with Symmetric Rank-One Non-negative Matrix Approximation , 2017, J. Optim. Theory Appl..

[5]  Haesun Park,et al.  SymNMF: nonnegative low-rank approximation of a similarity matrix for graph clustering , 2014, Journal of Global Optimization.

[6]  Chris H. Q. Ding,et al.  Symmetric Nonnegative Matrix Factorization for Graph Clustering , 2012, SDM.

[7]  Yu-Jin Zhang,et al.  FastNMF: highly efficient monotonic fixed-point nonnegative matrix factorization algorithm with good applicability , 2009, J. Electronic Imaging.

[8]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[9]  A. Berman,et al.  Completely Positive Matrices , 2003 .

[10]  Nicolas Gillis.  Nonnegative matrix factorization : complexity, algorithms and applications , 2011 .

[11]  Joydeep Ghosh,et al.  Under Consideration for Publication in Knowledge and Information Systems Generative Model-based Document Clustering: a Comparative Study , 2003 .

[12]  Xiaohui Yan,et al.  Learning Topics in Short Texts by Non-negative Matrix Factorization on Term Correlation Matrix , 2013, SDM.

[13]  Jing Hua,et al.  Non-negative matrix factorization for semi-supervised data clustering , 2008, Knowledge and Information Systems.

[14]  Stephen J. Wright.  Coordinate descent algorithms , 2015, Mathematical Programming.

[15]  Inderjit S. Dhillon,et al.  Fast coordinate descent methods with variable selection for non-negative matrix factorization , 2011, KDD.

[16]  Samuel Burer,et al.  On the copositive representation of binary and continuous nonconvex quadratic programs , 2009, Math. Program..

[17]  Nicolas Gillis,et al.  Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization , 2011, Neural Computation.

[18]  Haesun Park,et al.  Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons , 2011, SIAM J. Sci. Comput..

[19]  Girolamo Cardano,et al.  Ars magna or The rules of algebra , 1993 .

[20]  Stephen P. Boyd,et al.  Generalized Low Rank Models , 2014, Found. Trends Mach. Learn..

[21]  Philip S. Yu,et al.  Relational clustering by symmetric convex coding , 2007, ICML '07.

[22]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[23]  Zhaoshui He,et al.  Symmetric Nonnegative Matrix Factorization: Algorithms and Applications to Probabilistic Clustering , 2011, IEEE Transactions on Neural Networks.

[24]  Haesun Park,et al.  Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[25]  Amnon Shashua,et al.  A unifying approach to hard and probabilistic clustering , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[26]  Upendra Prasad.  Nonnegative matrix factorization: Analysis, algorithms and applications , 2009 .

[27]  Erkki Oja,et al.  Quadratic nonnegative matrix factorization , 2012, Pattern Recognit..

[28]  Erkki Oja,et al.  Clustering by Nonnegative Matrix Factorization Using Graph Random Walk , 2012, NIPS.

[29]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[30]  R. Bro,et al.  Resolving the sign ambiguity in the singular value decomposition , 2008 .

[31]  Shang-Hua Teng,et al.  Spectral sparsification of graphs: theory and algorithms , 2013, CACM.

[32]  Shuzhong Zhang,et al.  Maximum Block Improvement and Polynomial Optimization , 2012, SIAM J. Optim..

[33]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.