Non-negative Matrix Factorization: A Survey

Non-negative matrix factorization (NMF) is a powerful tool for data science researchers, and it has been successfully applied to data mining and machine learning community, due to its advantages such as simple form, good interpretability and less storage space. In this paper, we give a detailed survey on existing NMF methods, including a comprehensive analysis of their design principles, characteristics and drawbacks. In addition, we also discuss various variants of NMF methods and analyse properties and applications of these variants. Finally, we evaluate the performance of nine NMF methods through numerical experiments, and the results show that NMF methods perform well in clustering tasks.

[1]  Cyril Ruckebusch,et al.  Application of a sparseness constraint in multivariate curve resolution - Alternating least squares. , 2018, Analytica chimica acta.

[2]  Honghui Fan,et al.  Deep semi-nonnegative matrix factorization with elastic preserving for data representation , 2020, Multimedia Tools and Applications.

[3]  Haesun Park,et al.  An Alternating Rank-K Nonnegative Least Squares Framework (ARkNLS) for Nonnegative Matrix Factorization , 2020, SIAM J. Matrix Anal. Appl..

[4]  Mengchu Zhou,et al.  Symmetric Nonnegative Matrix Factorization-Based Community Detection Models and Their Convergence Analysis , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Daoqiang Zhang,et al.  Non-negative Matrix Factorization on Kernels , 2006, PRICAI.

[6]  Nicolas Gillis,et al.  Two algorithms for orthogonal nonnegative matrix factorization with application to clustering , 2012, Neurocomputing.

[7]  Rahil Garnavi,et al.  Skin Disease Recognition Using Deep Saliency Features and Multimodal Learning of Dermoscopy and Clinical Images , 2017, MICCAI.

[8]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[9]  D. Shen,et al.  Joint prediction and time estimation of COVID-19 developing severe symptoms using chest CT scan , 2020, Medical Image Analysis.

[10]  Qingfu Zhang,et al.  Self-Supervised Symmetric Nonnegative Matrix Factorization , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Zhenhua Guo,et al.  Loss-Based Attention for Interpreting Image-Level Prediction of Convolutional Neural Networks , 2020, IEEE Transactions on Image Processing.

[12]  Haesun Park,et al.  Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons , 2011, SIAM J. Sci. Comput..

[13]  Karthik Devarajan,et al.  Non-negative matrix factorization based on generalized dual divergence , 2019, ArXiv.

[14]  Blake Hunter,et al.  A Deep Non-Negative Matrix Factorization Neural Network , 2017 .

[15]  Xiaojie Su,et al.  Robust Manhattan non-negative matrix factorization for image recovery and representation , 2020, Inf. Sci..

[16]  Badong Chen,et al.  Robust orthogonal nonnegative matrix tri-factorization for data representation , 2020, Knowl. Based Syst..

[17]  Heng Tao Shen,et al.  Heterogeneous data fusion for predicting mild cognitive impairment conversion , 2021, Inf. Fusion.

[18]  Xiaofeng Zhu,et al.  Efficient Utilization of Missing Data in Cost-Sensitive Learning , 2019, IEEE Transactions on Knowledge and Data Engineering.

[19]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[20]  Seokjin Lee,et al.  Multichannel non-negative matrix factorisation based on alternating least squares for audio source separation system , 2015 .

[21]  Shichao Zhang,et al.  Robust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Jianfeng Lu,et al.  Community detection algorithm based on nonnegative matrix factorization and pairwise constraints , 2020 .

[24]  Xiaofeng Zhu,et al.  Unsupervised feature selection by self-paced learning regularization , 2020, Pattern Recognit. Lett..

[25]  Yue Gao,et al.  Inductive Multi-Hypergraph Learning and Its Application on View-Based 3D Object Classification , 2018, IEEE Transactions on Image Processing.

[26]  Aimin Zhou,et al.  Non-negative Matrix Factorization: A Short Survey on Methods and Applications , 2012, ISICA.

[27]  Yuzuru Tanaka,et al.  A Fast Hierarchical Alternating Least Squares Algorithm for Orthogonal Nonnegative Matrix Factorization , 2014, ACML.

[28]  Bob Zhang,et al.  Similarity Learning-Induced Symmetric Nonnegative Matrix Factorization for Image Clustering , 2019, IEEE Access.

[29]  Chris H. Q. Ding,et al.  Nonnegative Matrix Factorizations for Clustering: A Survey , 2018, Data Clustering: Algorithms and Applications.

[30]  Marc Teboulle,et al.  Novel Proximal Gradient Methods for Nonnegative Matrix Factorization with Sparsity Constraints , 2020, SIAM J. Imaging Sci..

[31]  Chuhsing Kate Hsiao,et al.  Network hub-node prioritization of gene regulation with intra-network association , 2020, BMC Bioinformatics.

[32]  Xuelong Li,et al.  Subspace clustering guided convex nonnegative matrix factorization , 2018, Neurocomputing.

[33]  Inderjit S. Dhillon,et al.  Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem , 2007, SDM.

[34]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[35]  Laurent Pueyo,et al.  Non-negative Matrix Factorization: Robust Extraction of Extended Structures , 2017, ArXiv.

[36]  Qiong Zhang,et al.  Network Embedding Using Semi-Supervised Kernel Nonnegative Matrix Factorization , 2019, IEEE Access.

[37]  S. Nemec,et al.  A principal components analysis: how pneumatization and edentulism contribute to maxillary atrophy. , 2017, Oral diseases.

[38]  Yun Liang,et al.  Anchor-Based Self-Ensembling for Semi-Supervised Deep Pairwise Hashing , 2020, International Journal of Computer Vision.

[39]  Bo Yu,et al.  A new kernel method for nonnegative matrix factorization , 2019, 2019 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA).

[40]  Les E. Atlas,et al.  Deep recurrent NMF for speech separation by unfolding iterative thresholding , 2017, 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA).

[41]  Jesús Bobadilla,et al.  Recommender Systems Clustering Using Bayesian Non Negative Matrix Factorization , 2018, IEEE Access.

[42]  Xuelong Li,et al.  Discriminative and Orthogonal Subspace Constraints-Based Nonnegative Matrix Factorization , 2018, ACM Trans. Intell. Syst. Technol..

[43]  Xuelong Li,et al.  Graph PCA Hashing for Similarity Search , 2017, IEEE Transactions on Multimedia.

[44]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[45]  Sam Kwong,et al.  Semisupervised Adaptive Symmetric Non-Negative Matrix Factorization , 2020, IEEE Transactions on Cybernetics.

[46]  Xinhe Xu,et al.  Facial expression recognition based on PCA and NMF , 2008, 2008 7th World Congress on Intelligent Control and Automation.

[47]  Zhenni Li,et al.  Uniform Distribution Non-Negative Matrix Factorization for Multiview Clustering , 2020, IEEE Transactions on Cybernetics.

[48]  S. Mifrah,et al.  Topic Modeling Coherence: A Comparative Study between LDA and NMF Models using COVID’19 Corpus , 2020 .

[49]  Farid Melgani,et al.  NMF with feature relationship preservation penalty term for clustering problems , 2021, Pattern Recognit..

[50]  Jonathan Le Roux,et al.  Deep NMF for speech separation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[51]  Philip S. Yu,et al.  A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning , 2021, IEEE Transactions on Knowledge and Data Engineering.

[52]  Jian Pei,et al.  High-Order Proximity Preserved Embedding for Dynamic Networks , 2018, IEEE Transactions on Knowledge and Data Engineering.

[53]  Xuelong Li,et al.  Refined-Graph Regularization-Based Nonnegative Matrix Factorization , 2017, ACM Trans. Intell. Syst. Technol..

[54]  Haizhou Li,et al.  Non-negative matrix factorization using stable alternating direction method of multipliers for source separation , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[55]  Dun Liu,et al.  A matrix factorization based dynamic granularity recommendation with three-way decisions , 2020, Knowl. Based Syst..

[56]  Domonkos Tikk,et al.  Alternating least squares for personalized ranking , 2012, RecSys.

[57]  Yannig Goude,et al.  Nonnegative Matrix Factorization with Side Information for Time Series Recovery and Prediction , 2017, IEEE Transactions on Knowledge and Data Engineering.

[58]  Lin Yang,et al.  Pairwise based deep ranking hashing for histopathology image classification and retrieval , 2018, Pattern Recognit..

[59]  Naixue Xiong,et al.  Deep Matrix Factorization With Implicit Feedback Embedding for Recommendation System , 2019, IEEE Transactions on Industrial Informatics.

[60]  Hassan Ghassemian,et al.  Spectral Unmixing of Hyperspectral Imagery Using Multilayer NMF , 2014, IEEE Geoscience and Remote Sensing Letters.

[61]  Nam Soo Kim,et al.  NMF-Based Speech Enhancement Using Bases Update , 2015, IEEE Signal Processing Letters.

[62]  Paul C. Boutros,et al.  Optimization and expansion of non-negative matrix factorization , 2020, BMC Bioinformatics.

[63]  Wen Zhang,et al.  A class of modified FR conjugate gradient method and applications to non-negative matrix factorization , 2017, Comput. Math. Appl..

[64]  Wei Liu,et al.  Supervised context-aware non-negative matrix factorization to handle high-dimensional high-correlated imbalanced biomedical data , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[65]  J. Eggert,et al.  Sparse coding and NMF , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[66]  Chris H. Q. Ding,et al.  Symmetric Nonnegative Matrix Factorization for Graph Clustering , 2012, SDM.

[67]  Nicolas Gillis,et al.  Accelerating Nonnegative Matrix Factorization Algorithms Using Extrapolation , 2018, Neural Computation.

[68]  Zhaoshui He,et al.  Symmetric Nonnegative Matrix Factorization: Algorithms and Applications to Probabilistic Clustering , 2011, IEEE Transactions on Neural Networks.

[69]  Zhenhua Guo,et al.  A Scalable Optimization Mechanism for Pairwise Based Discrete Hashing , 2021, IEEE Transactions on Image Processing.

[70]  Dinggang Shen,et al.  Learning longitudinal classification-regression model for infant hippocampus segmentation , 2020, Neurocomputing.

[71]  Simon J. L. Billinge,et al.  A fast two-stage algorithm for non-negative matrix factorization in streaming data , 2021, ArXiv.

[72]  Chris H. Q. Ding,et al.  Robust nonnegative matrix factorization using L21-norm , 2011, CIKM '11.

[73]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[74]  Wei Zheng,et al.  Spectral rotation for deep one-step clustering , 2020, Pattern Recognit..

[75]  Wenju Liu,et al.  Deep Learning Based Speech Separation via NMF-Style Reconstructions , 2018, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[76]  Erkki Oja,et al.  Projective Nonnegative Matrix Factorization for Image Compression and Feature Extraction , 2005, SCIA.

[77]  Huifang Ma,et al.  Orthogonal Nonnegative Matrix Tri-factorization for Semi-supervised Document Co-clustering , 2010, PAKDD.

[78]  Nicolas Gillis,et al.  Nonnegative Factorization and The Maximum Edge Biclique Problem , 2008, 0810.4225.

[79]  George Trigeorgis,et al.  A Deep Semi-NMF Model for Learning Hidden Representations , 2014, ICML.

[80]  Shuyuan Yang,et al.  Feature selection based dual-graph sparse non-negative matrix factorization for local discriminative clustering , 2018, Neurocomputing.

[81]  Xuelong Li,et al.  Block-Row Sparse Multiview Multilabel Learning for Image Classification , 2016, IEEE Transactions on Cybernetics.

[82]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[83]  Masataka Goto,et al.  Beyond NMF: Time-Domain Audio Source Separation without Phase Reconstruction , 2013, ISMIR.

[84]  Ying Zhang,et al.  Attention-Based Multi-NMF Deep Neural Network with Multimodality Data for Breast Cancer Prognosis Model , 2019, BioMed research international.

[85]  Shichao Zhang,et al.  Spectral clustering via half-quadratic optimization , 2019, World Wide Web.

[86]  Junyu Dong,et al.  Change Detection in SAR Images Based on Deep Semi-NMF and SVD Networks , 2017, Remote. Sens..

[87]  M. Ortiz,et al.  A material‐independent method for extending stress update algorithms from small-strain plasticity to finite plasticity with multiplicative kinematics , 1992 .

[88]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[89]  Xiaofeng Zhu,et al.  One-Step Multi-View Spectral Clustering , 2019, IEEE Transactions on Knowledge and Data Engineering.

[90]  Jia Wang,et al.  A Regularized Convex Nonnegative Matrix Factorization Model for signed network analysis , 2021, Social Network Analysis and Mining.

[91]  Andrzej Cichocki,et al.  Regularized Alternating Least Squares Algorithms for Non-negative Matrix/Tensor Factorization , 2007, ISNN.

[92]  Shenglan Liu,et al.  Deep graph convolution neural network with non-negative matrix factorization for community discovery , 2021, ArXiv.

[93]  Moses Charikar,et al.  Approximation Algorithms for Orthogonal Non-negative Matrix Factorization , 2021, AISTATS.

[94]  Guorong Wu,et al.  Brain functional connectivity analysis based on multi-graph fusion , 2021, Medical Image Anal..

[95]  Zhenhua Guo,et al.  Two-Dimensional Whitening Reconstruction for Enhancing Robustness of Principal Component Analysis , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[96]  Pascal Fernsel,et al.  Supervised non-negative matrix factorization methods for MALDI imaging applications , 2018, Bioinform..

[97]  Jane You,et al.  Hyperspectral image unsupervised classification by robust manifold matrix factorization , 2019, Inf. Sci..