Regularized NNLS Algorithms for Nonnegative Matrix Factorization with Application to Text Document Clustering

Nonnegative Matrix Factorization (NMF) has recently received much attention both in an algorithmic aspect as well as in applications. Text document clustering and supervised classification are important applications of NMF. Various types of numerical optimization algorithms have been proposed for NMF, which includes multiplicative, projected gradient descent, alternating least squares and active-set ones. In this paper, we discuss the selected Non-Negatively constrained Least Squares (NNLS) algorithms (a family of the NNLS algorithm proposed by Lawson and Hanson) that belong to a class of active-set methods. We noticed that applying the NNLS algorithm to the Tikhonov regularized LS objective function with a regularization parameter exponentially decreasing considerably increases the accuracy of data clustering as well as it reduces the risk of getting stuck into unfavorable local minima. Moreover, the experiments demonstrate that the regularized NNLS algorithm is superior to many well-known NMF algorithms used for text document clustering.

[1]  Chris H. Q. Ding,et al.  Nonnegative Matrix Factorization and Probabilistic Latent Semantic Indexing: Equivalence Chi-Square Statistic, and a Hybrid Method , 2006, AAAI.

[2]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[3]  Ioannis Pitas,et al.  Application of non-negative and local non negative matrix factorization to facial expression recognition , 2004, ICPR 2004.

[4]  Qian Du,et al.  Dependent component analysis for blind restoration of images degraded by turbulent atmosphere , 2009, Neurocomputing.

[5]  Hujun Bao,et al.  Understanding the Power of Clause Learning , 2009, IJCAI.

[6]  Tao Li,et al.  The Relationships Among Various Nonnegative Matrix Factorization Methods for Clustering , 2006, Sixth International Conference on Data Mining (ICDM'06).

[7]  Lucas C. Parra,et al.  Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain , 2004, IEEE Transactions on Medical Imaging.

[8]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Christoph Schnörr,et al.  Learning Sparse Representations by Non-Negative Matrix Factorization and Sequential Cone Programming , 2006, J. Mach. Learn. Res..

[10]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[11]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[12]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[13]  M. V. Van Benthem,et al.  Fast algorithm for the solution of large‐scale non‐negativity‐constrained least squares problems , 2004 .

[14]  Barak A. Pearlmutter,et al.  Convolutive Non-Negative Matrix Factorisation with a Sparseness Constraint , 2006, 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing.

[15]  S. Sra.  Nonnegative Matrix Approximation: Algorithms and Applications , 2006 .

[16]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[17]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[18]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[19]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[20]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[21]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[22]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[23]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .