Sparse and unique nonnegative matrix factorization through data preprocessing

Nonnegative matrix factorization (NMF) has become a very popular technique in machine learning because it automatically extracts meaningful features through a sparse and part-based representation. However, NMF has the drawback of being highly ill-posed, that is, there typically exist many different but equivalent factorizations. In this paper, we introduce a completely new way to obtaining more well-posed NMF problems whose solutions are sparser. Our technique is based on the preprocessing of the nonnegative input data matrix, and relies on the theory of M-matrices and the geometric interpretation of NMF. This approach provably leads to optimal and sparse solutions under the separability assumption of Donoho and Stodden (2003), and, for rank-three matrices, makes the number of exact factorizations finite. We illustrate the effectiveness of our technique on several image data sets.

[1]  Maurice D. Craig,et al.  Minimum-volume transforms for remotely sensed data , 1994, IEEE Trans. Geosci. Remote. Sens..

[2]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[3]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[4]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[5]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[6]  Chong-Yung Chi,et al.  Nonnegative Least-Correlated Component Analysis for Separation of Dependent Sources by Volume Maximization , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Toshihisa Tanaka,et al.  First results on uniqueness of sparse non-negative matrix factorization , 2005, 2005 13th European Signal Processing Conference.

[8]  Efstratios Gallopoulos,et al.  Computing symmetric nonnegative rank factorizations , 2012 .

[9]  Nicolas Gillis,et al.  Using underapproximations for sparse nonnegative matrix factorization , 2009, Pattern Recognit..

[10]  Jack Xin,et al.  Underdetermined Sparse Blind Source Separation of Nonnegative and Partially Overlapped Data , 2011, SIAM J. Sci. Comput..

[11]  Zhaoshui He,et al.  Minimum-Volume-Constrained Nonnegative Matrix Factorization: Enhanced Ability of Learning Parts , 2011, IEEE Transactions on Neural Networks.

[12]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[13]  Jiu Ding,et al.  Elementary Properties of Nonnegative Matrices , 2009 .

[14]  Moody T. Chu,et al.  Low-Dimensional Polytope Approximation and Its Applications to Nonnegative Matrix Factorization , 2008, SIAM J. Sci. Comput..

[15]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[16]  Boris T. Polyak,et al.  Hard Problems in Linear Control Theory: Possible Approaches to Solution , 2005 .

[17]  Andrzej Cichocki,et al.  Non-Negative Matrix Factorization , 2020 .

[18]  Nicolas Gillis,et al.  Nonnegative Factorization and The Maximum Edge Biclique Problem , 2008, 0810.4225.

[19]  C. Ding,et al.  On the Equivalence of Nonnegative Matrix Factorization and K-means - Spectral Clustering , 2005 .

[20]  Nicolas Gillis.  Nonnegative matrix factorization : complexity, algorithms and applications , 2011 .

[21]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[22]  James Curry,et al.  Non-negative matrix factorization: Ill-posedness and a geometric algorithm , 2009, Pattern Recognit..

[23]  Nicolas Gillis,et al.  Fast and Robust Recursive Algorithmsfor Separable Nonnegative Matrix Factorization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[25]  Alok Aggarwal,et al.  Finding Minimal Convex Nested Polygons , 1989, Inf. Comput..

[26]  Hairong Qi,et al.  Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Nancy Bertin,et al.  Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music Analysis , 2009, Neural Computation.

[28]  Michiel Hazewinkel.  On positive vectors, positive matrices and the specialization order , 1984 .

[29]  L. B. Thomas.  Rank Factorization of Nonnegative Matrices (A. Berman) , 1974 .

[30]  Nicolas Gillis,et al.  Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization , 2011, Neural Computation.

[31]  Nicolas Gillis,et al.  Two algorithms for orthogonal nonnegative matrix factorization with application to clustering , 2012, Neurocomputing.

[32]  Fabio Rapallo,et al.  Perturbation of Matrices and Nonnegative Rank with a View toward Statistical Models , 2010, SIAM J. Matrix Anal. Appl..

[33]  Mireille Guillaume,et al.  Minimum Dispersion Constrained Nonnegative Matrix Factorization to Unmix Hyperspectral Data , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[35]  Nicolas Gillis,et al.  On the Geometric Interpretation of the Nonnegative Rank , 2010, 1009.0880.

[36]  O. Taussky.  A Recurring Theorem on Determinants , 1949 .

[37]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[38]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[39]  Mark D. Plumbley,et al.  Theorems on Positive Data: On the Uniqueness of NMF , 2008, Comput. Intell. Neurosci..

[40]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[41]  Christos Boutsidis,et al.  An improved approximation algorithm for the column subset selection problem , 2008, SODA.

[42]  Sanjeev Arora,et al.  Computing a nonnegative matrix factorization -- provably , 2011, STOC '12.

[43]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .