Exact and Heuristic Algorithms for Semi-Nonnegative Matrix Factorization

Given a matrix $M$ (not necessarily nonnegative) and a factorization rank $r$, semi-nonnegative matrix factorization (semi-NMF) looks for a matrix $U$ with $r$ columns and a nonnegative matrix $V$ with $r$ rows such that $UV$ is the best possible approximation of $M$ according to some metric. In this paper, we study the properties of semi-NMF from which we develop exact and heuristic algorithms. Our contribution is threefold. First, we prove that the error of a semi-NMF of rank $r$ has to be smaller than the best unconstrained approximation of rank $r-1$. This leads us to a new initialization procedure based on the singular value decomposition (SVD) with a guarantee on the quality of the approximation. Second, we propose an exact algorithm (that is, an algorithm that finds an optimal solution), also based on the SVD, for a certain class of matrices (including nonnegative irreducible matrices) from which we derive an initialization for matrices not belonging to that class. Numerical experiments illustrate that this second approach performs extremely well, and allows us to compute optimal semi-NMF decompositions in many situations. Finally, we analyze the computational complexity of semi-NMF proving its NP-hardness, already in the rank-one case (that is, for $r = 1$), and we show that semi-NMF is sometimes ill-posed (that is, an optimal solution does not exist).

[1]  Aline Roumy,et al.  Neighbor embedding based single-image super-resolution using Semi-Nonnegative Matrix Factorization , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Sanjeev Arora,et al.  Computational Complexity: A Modern Approach , 2009 .

[3]  Nicolas Gillis,et al.  Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization , 2011, Neural Computation.

[4]  Jieping Ye,et al.  Sparse non-negative tensor factorization using columnwise coordinate descent , 2012, Pattern Recognit..

[5]  J. Aplevich,et al.  Lecture Notes in Control and Information Sciences , 1979 .

[6]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[7]  Nicolas Gillis,et al.  On the Geometric Interpretation of the Nonnegative Rank , 2010, 1009.0880.

[8]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[9]  M. Hanafi,et al.  Semi-nonnegative rank for real matrices and its connection to the usual rank , 2015 .

[10]  George Trigeorgis,et al.  A Deep Semi-NMF Model for Learning Hidden Representations , 2014, ICML.

[11]  Bruce A. Draper,et al.  Semi-Nonnegative Matrix Factorization for Motion Segmentation with Missing Data , 2012, ECCV.

[12]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[13]  J. Edmonds.  Systems of distinct representatives and linear algebra , 1967 .

[14]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[15]  S. Vavasis.  Nonlinear optimization: complexity issues , 1991 .

[16]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[17]  Nicolas Gillis,et al.  A continuous characterization of the maximum-edge biclique problem , 2014, J. Glob. Optim..

[18]  Naoto Yokoya,et al.  Generalized bilinear model based nonlinear unmixing using semi-nonnegative matrix factorization , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[19]  Inderjit S. Dhillon,et al.  Fast coordinate descent methods with variable selection for non-negative matrix factorization , 2011, KDD.

[20]  Dimitri P. Bertsekas.  Corrections for the book NONLINEAR PROGRAMMING: 2ND , 2005 .

[21]  Lenore Blum,et al.  Computing over the Reals: Where Turing Meets Newton , 2004 .

[22]  R. Bro,et al.  Resolving the sign ambiguity in the singular value decomposition , 2008 .

[23]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  P. Strevens.  Iii , 1985 .

[25]  Yu-Jin Zhang,et al.  FastNMF: highly efficient monotonic fixed-point nonnegative matrix factorization algorithm with good applicability , 2009, J. Electronic Imaging.

[26]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..