Similarity Learning-Induced Symmetric Nonnegative Matrix Factorization for Image Clustering

As a typical variation of nonnegative matrix factorization (NMF), symmetric NMF (SNMF) is capable of exploiting information of the cluster embedded in the matrix of similarity. The traditional SNMF-based methods for clustering first performs the techniques of the similarity learning on input data to learn a matrix of similarity, which is subsequently factorized by SNMF or one of its variants to learn information from the cluster. While these methods have led to satisfactory clustering results, it is suboptimal, since they do not explicitly exploit the fact that processes of the similarity learning and the clustering are depend on each other. In this paper, we describe a new SNMF model, termed similarity learning-induced SNMF (SLSNMF). SLSNMF can be considered as a unified framework that jointly considers these two processes. SLSNMF improves the clustering performance of SNMF by thoroughly exploring the mutual reinforcement between the process of similarity learning and the process of clustering until convergence. We incorporate a constraint into the standard SNMF model to learn the matrices of similarity and cluster simultaneously. Meanwhile, for solving this new model, we use the strategy of alternating iterative and derive an efficient algorithm, whose convergence is theoretically guaranteed. Experimental results over three benchmark image data sets demonstrate that SLSNMF outperforms the state-of-the-art methods for clustering.

[1]  Jiawei Han,et al.  Locally Consistent Concept Factorization for Document Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[2]  Haesun Park,et al.  SymNMF: nonnegative low-rank approximation of a similarity matrix for graph clustering , 2014, Journal of Global Optimization.

[3]  Jiebo Luo,et al.  Constrained Clustering With Nonnegative Matrix Factorization , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Zenglin Xu,et al.  Clustering with Similarity Preserving , 2019, Neurocomputing.

[5]  Wai Keung Wong,et al.  Approximate Low-Rank Projection Learning for Feature Extraction , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Chunwei Tian,et al.  Low-rank representation with adaptive graph regularization , 2018, Neural Networks.

[7]  Sam Kwong,et al.  Pairwise Constraint Propagation-Induced Symmetric Nonnegative Matrix Factorization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[9]  Erkki Oja,et al.  Linear and Nonlinear Projective Nonnegative Matrix Factorization , 2010, IEEE Transactions on Neural Networks.

[10]  Feiping Nie,et al.  An Iterative Locally Linear Embedding Algorithm , 2012, ICML.

[11]  Yu Zhang,et al.  A Fast Non-Smooth Nonnegative Matrix Factorization for Learning Sparse Representation , 2016, IEEE Access.

[12]  Yun Fu,et al.  Consensus Guided Multi-View Clustering , 2018, ACM Trans. Knowl. Discov. Data.

[13]  Jinhui Tang,et al.  Robust Structured Nonnegative Matrix Factorization for Image Representation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Wai Keung Wong,et al.  Robust Latent Subspace Learning for Image Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Songtao Lu,et al.  A nonconvex splitting method for symmetric nonnegative matrix factorization: Convergence analysis and optimality , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[17]  Zhao Kang,et al.  Kernel-driven similarity learning , 2017, Neurocomputing.

[18]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[19]  Lunke Fei,et al.  Robust Sparse Linear Discriminant Analysis , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Hongchuan Yu,et al.  Diverse Non-Negative Matrix Factorization for Multiview Data Representation , 2018, IEEE Transactions on Cybernetics.

[21]  Xiaobing Pei,et al.  Joint Sparse Representation and Embedding Propagation Learning: A Framework for Graph-Based Semisupervised Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[23]  Tao Li,et al.  The Relationships Among Various Nonnegative Matrix Factorization Methods for Clustering , 2006, Sixth International Conference on Data Mining (ICDM'06).

[24]  Zenglin Xu,et al.  Multi-graph Fusion for Multi-view Spectral Clustering , 2019, Knowl. Based Syst..

[25]  Lunke Fei,et al.  Low-Rank Preserving Projection Via Graph Regularized Reconstruction , 2019, IEEE Transactions on Cybernetics.

[26]  Erkki Oja,et al.  Low-Rank Doubly Stochastic Matrix Decomposition for Cluster Analysis , 2016, J. Mach. Learn. Res..

[27]  Hong Liu,et al.  Incomplete Multiview Spectral Clustering With Adaptive Graph Learning , 2020, IEEE Transactions on Cybernetics.

[28]  Larry S. Davis,et al.  Truncated Cauchy Non-Negative Matrix Factorization , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Xuelong Li,et al.  Learning a Nonnegative Sparse Graph for Linear Regression , 2015, IEEE Transactions on Image Processing.

[30]  Xuelong Li,et al.  Robust Semi-Supervised Subspace Clustering via Non-Negative Low-Rank Representation , 2016, IEEE Transactions on Cybernetics.

[31]  Kai Liu,et al.  Dropping Symmetry for Fast Symmetric Nonnegative Matrix Factorization , 2018, NeurIPS.

[32]  Prabhu Babu,et al.  Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning , 2017, IEEE Transactions on Signal Processing.

[33]  Wei Yan,et al.  A novel regularized concept factorization for document clustering , 2017, Knowl. Based Syst..

[34]  Xinbo Gao,et al.  Semi-Supervised Nonnegative Matrix Factorization via Constraint Propagation , 2016, IEEE Transactions on Cybernetics.

[35]  Yong Xiang,et al.  Non-Negative Matrix Factorization With Dual Constraints for Image Clustering , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[36]  Chris H. Q. Ding,et al.  Simultaneous clustering of multi-type relational data via symmetric nonnegative matrix tri-factorization , 2011, CIKM '11.

[37]  Zhaoshui He,et al.  Symmetric Nonnegative Matrix Factorization: Algorithms and Applications to Probabilistic Clustering , 2011, IEEE Transactions on Neural Networks.

[38]  Zenglin Xu,et al.  Low-rank kernel learning for graph-based clustering , 2019, Knowl. Based Syst..

[39]  Xuelong Li,et al.  Flexible Affinity Matrix Learning for Unsupervised and Semisupervised Classification , 2019, IEEE Transactions on Neural Networks and Learning Systems.