Semi-Supervised Multi-view clustering based on orthonormality-constrained nonnegative matrix factorization

Abstract Multi-view clustering aims at integrating the complementary information between different views so as to obtain an accurate clustering result. In addition, the traditional clustering is a kind of unsupervised learning method, which does not take the label information into learning. In this paper, we propose a novel model, called semi-supervised multi-view clustering based on orthonormality-constrained nonnegative matrix factorization (MVOCNMF), to cluster the multi-view data into a number of categories. In the proposed model, based on the label information, we first learn the low-dimensional representations of data by the constrained NMF technique, and simultaneously cluster the samples with the same label into the clustering prototypes for each view. After that, we put forward a novel orthonormality constraint term to obtain the desirable representations for each view, and use the co-regularization to integrate the complementary information from different views. We further develop an alternating minimization algorithm to solve the proposed model, and present the convergence analysis and computational complexity of the proposed method. Extensive experimental results on several multi-view datasets have shown that the proposed MVOCNMF method outperforms the existing multi-view clustering methods.

[1]  Tao Mei,et al.  Towards Cross-Domain Learning for Social Video Popularity Prediction , 2013, IEEE Transactions on Multimedia.

[2]  Sharma Chakravarthy,et al.  Clustering data streams using grid-based synopsis , 2013, Knowledge and Information Systems.

[3]  Ming Li,et al.  Feature extraction via multi-view non-negative matrix factorization with local graph regularization , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[4]  H. Sebastian Seung,et al.  The Manifold Ways of Perception , 2000, Science.

[5]  Ling Li,et al.  Fuzzy based affinity learning for spectral clustering , 2016, Pattern Recognit..

[6]  Julien Jacques,et al.  Model-based co-clustering for functional data , 2016, Neurocomputing.

[7]  Jing Zhang,et al.  A Grid-Based Clustering Algorithm via Load Analysis for Industrial Internet of Things , 2018, IEEE Access.

[8]  Mohamed Nadif,et al.  Sparse Tensor Co-clustering as a Tool for Document Categorization , 2019, SIGIR.

[9]  Jialie Shen,et al.  On Effective Location-Aware Music Recommendation , 2016, ACM Trans. Inf. Syst..

[10]  Min-Yen Kan,et al.  Comment-based multi-view clustering of web 2.0 items , 2014, WWW.

[11]  Qinghua Hu,et al.  Generalized Latent Multi-View Subspace Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Lei Wang,et al.  Multiple Kernel Clustering with Local Kernel Alignment Maximization , 2016, IJCAI.

[13]  Z. Jane Wang,et al.  ICFS Clustering With Multiple Representatives for Large Data , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Anna Goldenberg,et al.  EquiNMF: Graph Regularized Multiview Nonnegative Matrix Factorization , 2014, ArXiv.

[15]  Bart De Moor,et al.  Gene prioritization and clustering by multi-view text mining , 2010, BMC Bioinformatics.

[16]  Dirk Kraft,et al.  Rotational Subgroup Voting and Pose Clustering for Robust 3D Object Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Cai Xu,et al.  Deep Multi-View Concept Learning , 2018, IJCAI.

[18]  René Vidal,et al.  Structured Sparse Subspace Clustering: A Joint Affinity Learning and Subspace Clustering Framework , 2016, IEEE Transactions on Image Processing.

[19]  Daniel A. Spielman,et al.  Spectral Graph Theory and its Applications , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[20]  Xuelong Li,et al.  Self-weighted spectral clustering with parameter-free constraint , 2017, Neurocomputing.

[21]  Meng Liu,et al.  Online Data Organizer: Micro-Video Categorization by Structure-Guided Multimodal Dictionary Learning , 2019, IEEE Transactions on Image Processing.

[22]  Z. Jane Wang,et al.  Unsupervised Multiview Nonnegative Correlated Feature Learning for Data Clustering , 2018, IEEE Signal Processing Letters.

[23]  Peter Meer,et al.  Semi-Supervised Kernel Mean Shift Clustering , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Johan A. K. Suykens,et al.  Multi-View Kernel Spectral Clustering , 2018, Inf. Fusion.

[25]  Lan Bai,et al.  Clustering by twin support vector machine and least square twin support vector classifier with uniform output coding , 2019, Knowl. Based Syst..

[26]  Hojjat Adeli,et al.  Permutation Jaccard Distance-Based Hierarchical Clustering to Estimate EEG Network Density Modifications in MCI Subjects , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Hao Wang,et al.  Multi-view Clustering via Concept Factorization with Local Manifold Regularization , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[28]  Weihua Ou,et al.  Multi-view non-negative matrix factorization by patch alignment framework with view consistency , 2016, Neurocomputing.

[29]  Yong Dou,et al.  Optimal Neighborhood Kernel Clustering with Multiple Kernels , 2017, AAAI.

[30]  Jianping Fan,et al.  Multi-View Concept Learning for Data Representation , 2015, IEEE Transactions on Knowledge and Data Engineering.

[31]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[32]  Xuelong Li,et al.  Self-weighted Multiview Clustering with Multiple Graphs , 2017, IJCAI.

[33]  Mohan S. Kankanhalli,et al.  Aspect-Aware Latent Factor Model: Rating Prediction with Ratings and Reviews , 2018, WWW.

[34]  Hongchuan Yu,et al.  Diverse Non-Negative Matrix Factorization for Multiview Data Representation , 2018, IEEE Transactions on Cybernetics.

[35]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[36]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Tao Zhou,et al.  Dual Shared-Specific Multiview Subspace Clustering , 2020, IEEE Transactions on Cybernetics.

[38]  Jiawei Han,et al.  Multi-View Clustering via Joint Nonnegative Matrix Factorization , 2013, SDM.

[39]  Hong Yu,et al.  Multi-view clustering via multi-manifold regularized non-negative matrix factorization , 2017, Neural Networks.

[40]  Xiao Wang,et al.  Adaptive Multi-view Semi-supervised Nonnegative Matrix Factorization , 2016, ICONIP.

[41]  Rong Wang,et al.  Fast spectral clustering learning with hierarchical bipartite graph for large-scale data , 2020, Pattern Recognit. Lett..

[42]  Hongsheng Li,et al.  A hierarchical image clustering cosegmentation framework , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Xuelong Li,et al.  Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours , 2017, AAAI.

[44]  Xiaochun Cao,et al.  Diversity-induced Multi-view Subspace Clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Lan Bai,et al.  Twin Support Vector Machine for Clustering , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Avory Bryant,et al.  RNN-DBSCAN: A Density-Based Clustering Algorithm Using Reverse Nearest Neighbor Density Estimates , 2018, IEEE Transactions on Knowledge and Data Engineering.

[47]  Rossano Schifanella,et al.  6 Seconds of Sound and Vision: Creativity in Micro-videos , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.