Adaptive Method for Nonsmooth Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is an emerging tool for meaningful low-rank matrix representation. In NMF, explicit constraints are usually required, such that NMF generates desired products (or factorizations), especially when the products have significant sparseness features. It is known that the ability of NMF in learning sparse representation can be improved by embedding a smoothness factor between the products. Motivated by this result, we propose an adaptive nonsmooth NMF (Ans-NMF) method in this paper. In our method, the embedded factor is obtained by using a data-related approach, so it matches well with the underlying products, implying a superior faithfulness of the representations. Besides, due to the usage of an adaptive selection scheme to this factor, the sparseness of the products can be separately constrained, leading to wider applicability and interpretability. Furthermore, since the adaptive selection scheme is processed through solving a series of typical linear programming problems, it can be easily implemented. Simulations using computer-generated data and real-world data show the advantages of the proposed Ans-NMF method over the state-of-the-art methods.

[1]  Michael J. Watts,et al.  IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS Publication Information , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Anastasios Tefas,et al.  Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification , 2006, IEEE Transactions on Neural Networks.

[3]  Zhenyue Zhang,et al.  Low-Rank Matrix Approximation with Manifold Regularization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[5]  Andrzej Cichocki,et al.  Fast Nonnegative Matrix/Tensor Factorization Based on Low-Rank Approximation , 2012, IEEE Transactions on Signal Processing.

[6]  Zhaoshui He,et al.  Symmetric Nonnegative Matrix Factorization: Algorithms and Applications to Probabilistic Clustering , 2011, IEEE Transactions on Neural Networks.

[7]  Emmanuel Vincent,et al.  Stability Analysis of Multiplicative Update Algorithms and Application to Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.

[8]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[9]  Dietrich Lehmann,et al.  Nonsmooth nonnegative matrix factorization (nsNMF) , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[11]  Yu-Jin Zhang,et al.  Nonnegative Matrix Factorization: A Comprehensive Review , 2013, IEEE Transactions on Knowledge and Data Engineering.

[12]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[13]  Yong Xiang,et al.  Nonnegative Blind Source Separation by Sparse Component Analysis Based on Determinant Measure , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Bo Du,et al.  An Endmember Dissimilarity Constrained Non-Negative Matrix Factorization Method for Hyperspectral Unmixing , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Patrik O. Hoyer,et al.  Modeling Receptive Fields with Non-Negative Sparse Coding , 2002, Neurocomputing.

[16]  Chris H. Q. Ding,et al.  Robust nonnegative matrix factorization using L21-norm , 2011, CIKM '11.

[17]  Jimeng Sun,et al.  Automatic Group Sparse Coding , 2011, AAAI.

[18]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Shengli Xie,et al.  Online Blind Source Separation Using Incremental Nonnegative Matrix Factorization With Volume Constraint , 2011, IEEE Transactions on Neural Networks.

[20]  Roy E. Marsten,et al.  Feature Article - Interior Point Methods for Linear Programming: Computational State of the Art , 1994, INFORMS J. Comput..

[21]  Yong Xiang,et al.  Projection-Pursuit-Based Method for Blind Separation of Nonnegative Sources , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Samy Bengio,et al.  Group Sparse Coding , 2009, NIPS.

[23]  Zhigang Luo,et al.  Online Nonnegative Matrix Factorization With Robust Stochastic Approximation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Naoto Yokoya,et al.  Nonlinear Unmixing of Hyperspectral Data Using Semi-Nonnegative Matrix Factorization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Gautham J. Mysore,et al.  Stopping Criteria for Non-Negative Matrix Factorization Based Supervised and Semi-Supervised Source Separation , 2014, IEEE Signal Processing Letters.

[26]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[27]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[28]  Sanjay Mehrotra,et al.  On the Implementation of a Primal-Dual Interior Point Method , 1992, SIAM J. Optim..

[29]  Lin Wu,et al.  A Fast Algorithm for Nonnegative Matrix Factorization and Its Convergence , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Andrzej Cichocki,et al.  Two Efficient Algorithms for Approximately Orthogonal Nonnegative Matrix Factorization , 2015, IEEE Signal Processing Letters.

[31]  Slim Essid,et al.  Smooth Nonnegative Matrix Factorization for Unsupervised Audiovisual Document Structuring , 2013, IEEE Transactions on Multimedia.

[32]  Zhaoshui He,et al.  Convolutive Blind Source Separation in the Frequency Domain Based on Sparse Representation , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[33]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[34]  Kyuwan Choi,et al.  Detecting the Number of Clusters in n-Way Probabilistic Clustering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Shengli Xie,et al.  Blind Spectral Unmixing Based on Sparse Nonnegative Matrix Factorization , 2011, IEEE Transactions on Image Processing.

[36]  Li Jia,et al.  Nonnegative Matrix Factorization With Regularizations , 2014, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[37]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[38]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[39]  Li Shang,et al.  Molecular Cancer Class Discovery Using Non-negative Matrix Factorization with Sparseness Constraint , 2007, ICIC.

[40]  A. Cichocki,et al.  Multilayer nonnegative matrix factorisation , 2006 .

[41]  Tao Wu,et al.  Automated Graph Regularized Projective Nonnegative Matrix Factorization for Document Clustering , 2014, IEEE Transactions on Cybernetics.

[42]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[43]  Anastasios Tefas,et al.  Multiplicative Update Rules for Concurrent Nonnegative Matrix Factorization and Maximum Margin Classification , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Chong-Yung Chi,et al.  Nonnegative Least-Correlated Component Analysis for Separation of Dependent Sources by Volume Maximization , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Fei Wang,et al.  Semi-Supervised Clustering via Matrix Factorization , 2008, SDM.

[46]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Wai Lok Woo,et al.  Variational Regularized 2-D Nonnegative Matrix Factorization , 2012, IEEE Transactions on Neural Networks and Learning Systems.