Robust nonnegative matrix factorization with structure regularization

Abstract Nonnegative matrix factorization (NMF) has attracted more and more attention due to its wide applications in computer vision, information retrieval, and machine learning. In contrast to the original NMF and its variants, this paper proposes a novel unsupervised learning framework, called robust structured nonnegative matrix factorization (RSNMF) which respects both global and local structures of the data space. Specifically, to learn a discriminative representation, RSNMF explores both the global structure via considering the data variance and the local structure via exploiting the data neighborhood. To well address the problem of noise and outliers, it imposes joint L2,1-norm minimization on both the loss function of NMF and the regularization of the basis matrix. The geometric structure and the joint L2,1-norm are formulated as an optimization model, which is solved by the proposed iterative algorithm. Finally, the convergence of RSNMF is analyzed theoretically and empirically. The experimental results on real-world data sets show the effectiveness of our proposed algorithm in comparison to state-of-the-art algorithms.

[1]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[2]  Lei Wang,et al.  Global and Local Structure Preservation for Feature Selection , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Kay Chen Tan,et al.  Multiobjective Sparse Non-Negative Matrix Factorization , 2019, IEEE Transactions on Cybernetics.

[4]  Honggang Zhang,et al.  Comments on "Globally Maximizing, Locally Minimizing: Unsupervised Discriminant Projection with Application to Face and Palm Biometrics" , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Meng Wang,et al.  Evolutionary nonnegative matrix factorization with adaptive control of cluster quality , 2018, Neurocomputing.

[6]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[7]  Zhigang Luo,et al.  Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent , 2011, IEEE Transactions on Image Processing.

[8]  Zhenyue Zhang,et al.  Low-Rank Matrix Approximation with Manifold Regularization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[11]  Di Zhang,et al.  Global plus local: A complete framework for feature extraction and recognition , 2014, Pattern Recognit..

[12]  Wei Chen,et al.  Manifold NMF with L21 norm for clustering , 2018, Neurocomputing.

[13]  Jinhui Tang,et al.  Robust Structured Nonnegative Matrix Factorization for Image Representation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Seungjin Choi,et al.  Manifold-respecting discriminant nonnegative matrix factorization , 2011, Pattern Recognit. Lett..

[15]  Xuelong Li,et al.  Refined-Graph Regularization-Based Nonnegative Matrix Factorization , 2017, ACM Trans. Intell. Syst. Technol..

[16]  David L. Sheinberg,et al.  Visual object recognition. , 1996, Annual review of neuroscience.

[17]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[18]  Erkki Oja,et al.  Linear and Nonlinear Projective Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.

[19]  Haipeng Peng,et al.  A Novel Digital Watermarking Based on General Non-Negative Matrix Factorization , 2018, IEEE Transactions on Multimedia.

[20]  Xiaojie Su,et al.  Robust Manhattan non-negative matrix factorization for image recovery and representation , 2020, Inf. Sci..

[21]  Jieping Ye,et al.  Integrating Global and Local Structures: A Least Squares Framework for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Mihai Datcu,et al.  Discriminative Nonnegative Matrix Factorization for dimensionality reduction , 2016, Neurocomputing.

[23]  Songcan Chen,et al.  Regularized soft K-means for discriminant analysis , 2013, Neurocomputing.

[24]  Witold Pedrycz,et al.  Global and local structure preserving sparse subspace learning: An iterative approach to unsupervised feature selection , 2015, Pattern Recognit..

[25]  Liang Tang,et al.  General subspace constrained non-negative matrix factorization for data representation , 2016, Neurocomputing.

[26]  Ivica Kopriva,et al.  A Nonlinear Orthogonal Non-Negative Matrix Factorization Approach to Subspace Clustering , 2017, Pattern Recognit..

[27]  Wai Lok Woo,et al.  Machine Learning Source Separation Using Maximum a Posteriori Nonnegative Matrix Factorization , 2014, IEEE Transactions on Cybernetics.

[28]  Jiebo Luo,et al.  Constrained Clustering With Nonnegative Matrix Factorization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[29]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[30]  Yong Xiang,et al.  Adaptive Method for Nonsmooth Nonnegative Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Jim Jing-Yan Wang,et al.  Multiple graph regularized nonnegative matrix factorization , 2013, Pattern Recognit..

[32]  D. Perrett,et al.  Recognition of objects and their component parts: responses of single units in the temporal cortex of the macaque. , 1994, Cerebral cortex.

[33]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[34]  Yong Chen,et al.  Soft orthogonal non-negative matrix factorization with sparse representation: Static and dynamic , 2018, Neurocomputing.

[35]  Guang Shi,et al.  Graph-based discriminative nonnegative matrix factorization with label information , 2017, Neurocomputing.

[36]  Feiping Nie,et al.  Fast Robust Non-Negative Matrix Factorization for Large-Scale Human Action Data Clustering , 2016, IJCAI.

[37]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[38]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Jing Liu,et al.  Robust Structured Subspace Learning for Data Representation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[41]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[42]  Lixiang Li,et al.  Incremental general non-negative matrix factorization without dimension matching constraints , 2018, Neurocomputing.

[43]  Tao Wu,et al.  Automated Graph Regularized Projective Nonnegative Matrix Factorization for Document Clustering , 2014, IEEE Transactions on Cybernetics.

[44]  Shaojie Qiao,et al.  Non-Negative Matrix Factorization With Locality Constrained Adaptive Graph , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Shengli Xie,et al.  Online Blind Source Separation Using Incremental Nonnegative Matrix Factorization With Volume Constraint , 2011, IEEE Transactions on Neural Networks.

[46]  Xuelong Li,et al.  Nonnegative Discriminant Matrix Factorization , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[47]  Xinbo Gao,et al.  Semi-Supervised Nonnegative Matrix Factorization via Constraint Propagation , 2016, IEEE Transactions on Cybernetics.

[48]  Deguang Kong,et al.  Elastic nonnegative matrix factorization , 2019, Pattern Recognit..

[49]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[50]  Dimitris S. Papailiopoulos,et al.  Orthogonal NMF through Subspace Exploration , 2015, NIPS.

[51]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[52]  Xuelong Li,et al.  Non-negative graph embedding , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Xuelong Li,et al.  Discriminative and Orthogonal Subspace Constraints-Based Nonnegative Matrix Factorization , 2018, ACM Trans. Intell. Syst. Technol..

[54]  Feiping Nie,et al.  Local Regression and Global Information-Embedded Dimension Reduction , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[55]  Lixiang Li,et al.  Sparse General Non-Negative Matrix Factorization Based on Left Semi-Tensor Product , 2019, IEEE Access.

[56]  Xuelong Li,et al.  Graph Regularized Non-Negative Low-Rank Matrix Factorization for Image Clustering , 2017, IEEE Transactions on Cybernetics.

[57]  Feiping Nie,et al.  Robust Manifold Nonnegative Matrix Factorization , 2014, ACM Trans. Knowl. Discov. Data.

[58]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Lixiang Li,et al.  Attributed community mining using joint general non-negative matrix factorization with graph Laplacian , 2018 .

[60]  Markus Flierl,et al.  Graph-Preserving Sparse Nonnegative Matrix Factorization With Application to Facial Expression Recognition , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[61]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[62]  Zhigang Luo,et al.  Online Nonnegative Matrix Factorization With Robust Stochastic Approximation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[63]  Stefanos Zafeiriou,et al.  Non-Negative Matrix Factorizations for Multiplex Network Analysis , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Songsong Wu,et al.  Uncorrelated slow feature discriminant analysis using globality preserving projections for feature extraction , 2015, Neurocomputing.

[65]  Anastasios Tefas,et al.  Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification , 2006, IEEE Transactions on Neural Networks.