Minimum-Volume-Constrained Nonnegative Matrix Factorization: Enhanced Ability of Learning Parts

Nonnegative matrix factorization (NMF) with minimum-volume-constraint (MVC) is exploited in this paper. Our results show that MVC can actually improve the sparseness of the results of NMF. This sparseness is L0-norm oriented and can give desirable results even in very weak sparseness situations, thereby leading to the significantly enhanced ability of learning parts of NMF. The close relation between NMF, sparse NMF, and the MVC_NMF is discussed first. Then two algorithms are proposed to solve the MVC_NMF model. One is called quadratic programming_MVC_NMF (QP_MVC_NMF) which is based on quadratic programming and the other is called negative glow_MVC_NMF (NG_MVC_NMF) because it uses multiplicative updates incorporating natural gradient ingeniously. The QP_MVC_NMF algorithm is quite efficient for small-scale problems and the NG_MVC_NMF algorithm is more suitable for large-scale problems. Simulations show the efficiency and validity of the proposed methods in applications of blind source separation and human face images analysis.

[1]  S. Amari,et al.  Nonnegative Matrix and Tensor Factorization [Lecture Notes] , 2008, IEEE Signal Processing Magazine.

[2]  Chong-Yung Chi,et al.  A convex analysis-based minimum-volume enclosing simplex algorithm for hyperspectral unmixing , 2009, IEEE Trans. Signal Process..

[3]  Jun Zhang,et al.  Nonorthogonal Approximate Joint Diagonalization With Well-Conditioned Diagonalizers , 2009, IEEE Transactions on Neural Networks.

[4]  Lars Kai Hansen,et al.  On Affine Non-Negative Matrix Factorization , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[5]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[6]  Anastasios Tefas,et al.  Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification , 2006, IEEE Transactions on Neural Networks.

[7]  Elmar Wolfgang Lang,et al.  Towards unique solutions of non-negative matrix factorization problems by a determinant criterion , 2011, Digit. Signal Process..

[8]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[9]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[10]  Chong-Yung Chi,et al.  Nonnegative Least-Correlated Component Analysis for Separation of Dependent Sources by Volume Maximization , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[12]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[14]  Andrzej Cichocki,et al.  New Algorithms for Non-Negative Matrix Factorization in Applications to Blind Source Separation , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[15]  J. Eggert,et al.  Sparse coding and NMF , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[16]  Ioannis Pitas,et al.  Nonnegative Matrix Factorization in Polynomial Feature Space , 2008, IEEE Transactions on Neural Networks.

[17]  Shengli Xie,et al.  Mixing Matrix Estimation From Sparse Mixtures With Unknown Number of Sources , 2011, IEEE Transactions on Neural Networks.

[18]  Shengli Xie,et al.  Online Blind Source Separation Using Incremental Nonnegative Matrix Factorization With Volume Constraint , 2011, IEEE Transactions on Neural Networks.

[19]  Christos Boutsidis,et al.  SVD based initialization: A head start for nonnegative matrix factorization , 2008, Pattern Recognit..

[20]  Toshihisa Tanaka,et al.  Blind Extraction of Global Signal From Multi-Channel Noisy Observations , 2010, IEEE Transactions on Neural Networks.

[21]  Fabian J. Theis,et al.  Sparse Nonnegative Matrix Factorization with Genetic Algorithms for Microarray Analysis , 2007, 2007 International Joint Conference on Neural Networks.

[22]  Stefan A. Robila,et al.  Considerations on Parallelizing Nonnegative Matrix Factorization for Hyperspectral Data Unmixing , 2009, IEEE Geoscience and Remote Sensing Letters.

[23]  Pierre Comon,et al.  Robust Independent Component Analysis by Iterative Maximization of the Kurtosis Contrast With Algebraic Optimal Step Size , 2010, IEEE Transactions on Neural Networks.

[24]  Nanning Zheng,et al.  Non-negative matrix factorization for visual coding , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[25]  Christoph Schnörr,et al.  Learning Sparse Representations by Non-Negative Matrix Factorization and Sequential Cone Programming , 2006, J. Mach. Learn. Res..

[26]  Erkki Oja,et al.  A "nonnegative PCA" algorithm for independent component analysis , 2004, IEEE Transactions on Neural Networks.

[27]  Chong-Yung Chi,et al.  A Convex Analysis Framework for Blind Separation of Non-Negative Sources , 2008, IEEE Transactions on Signal Processing.

[28]  Maurice D. Craig,et al.  Minimum-volume transforms for remotely sensed data , 1994, IEEE Trans. Geosci. Remote. Sens..

[29]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Liqing Zhang,et al.  Flexible Component Analysis for Sparse, Smooth, Nonnegative Coding or Representation , 2007, ICONIP.

[31]  Aapo Hyvärinen,et al.  The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis , 1999, Neural Processing Letters.

[32]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[33]  Toshihisa Tanaka,et al.  First results on uniqueness of sparse non-negative matrix factorization , 2005, 2005 13th European Signal Processing Conference.

[34]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[35]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[36]  Stefanos Zafeiriou,et al.  Discriminant Nonnegative Tensor Factorization Algorithms , 2009, IEEE Transactions on Neural Networks.

[37]  Ying Chen,et al.  Clustering-based initialization for non-negative matrix factorization , 2008, Appl. Math. Comput..

[38]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[39]  Erkki Oja,et al.  Linear and Nonlinear Projective Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.

[40]  Elmar Wolfgang Lang,et al.  Minimum Determinant Constraint for Non-negative Matrix Factorization , 2009, ICA.

[41]  Emmanuel Vincent,et al.  Stability Analysis of Multiplicative Update Algorithms and Application to Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.

[42]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with constrained second-order optimization , 2007, Signal Process..

[43]  Hao Shen,et al.  Local Convergence Analysis of FastICA and Related Algorithms , 2008, IEEE Transactions on Neural Networks.