Convex Nonnegative Matrix Factorization with Rank-1 Update for Clustering

In convex nonnegative matrix factorization, the feature vectors are modeled by convex combinations of observation vectors. In the paper, we propose to express the factorization model in terms of the sum of rank-1 matrices. Then the sparse factors can be easily estimated by applying the concept of the Hierarchical Alternating Least Squares (HALS) algorithm which is still regarded as one of the most effective algorithms for solving many nonnegative matrix factorization problems. The proposed algorithm has been applied to find partially overlapping clusters in various datasets, including textual documents. The experiments demonstrate the high performance of the proposed approach.

[1]  M. V. Van Benthem,et al.  Fast algorithm for the solution of large‐scale non‐negativity‐constrained least squares problems , 2004 .

[2]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with constrained second-order optimization , 2007, Signal Process..

[3]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[4]  Rafal Zdunek,et al.  Data Clustering with Semi-binary Nonnegative Matrix Factorization , 2006, ICAISC.

[5]  Andrzej Cichocki,et al.  Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization , 2007, ICA.

[6]  Lixing Han,et al.  Alternating projected Barzilai-Borwein methods for nonnegative matrix factorization. , 2009 .

[7]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[8]  Mireille Guillaume,et al.  HALS-based NMF with flexible constraints for hyperspectral unmixing , 2012, EURASIP J. Adv. Signal Process..

[9]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[10]  Erkki Oja,et al.  Linear and Nonlinear Projective Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.

[11]  Michael Möller,et al.  A Convex Model for Nonnegative Matrix Factorization and Dimensionality Reduction on Physical Space , 2011, IEEE Transactions on Image Processing.

[12]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[13]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Nicolas Gillis,et al.  Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization , 2011, Neural Computation.

[15]  Alexandre d'Aspremont,et al.  Convex Algorithms for Nonnegative Matrix Factorization , 2012 .

[16]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[17]  Haesun Park,et al.  Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons , 2011, SIAM J. Sci. Comput..

[18]  Christian Bauckhage,et al.  Convex Non-negative Matrix Factorization in the Wild , 2009, 2009 Ninth IEEE International Conference on Data Mining.