Deep graph regularized non-negative matrix factorization for multi-view clustering

Abstract Multi-view clustering is an unsupervised method which aims to enhance the clustering performance by combining the knowledge from multiple view data. Non-negative matrix factorization (NMF) is one of the most favourable multi-view clustering methods due to its strong representation ability of non-negative data. However, NMF only factorizes the data matrix into two non-negative factor matrices, which may limit its ability to learn higher level and more complex hierarchical information. To overcome this shortcoming, in this paper, we propose a multi-view clustering method based on deep graph regularized non-negative matrix factorization (MvDGNMF). MvDGNMF is able to extract more abstract representation by constructing a multilayer NMF model with graph Laplacian regularization and drive the last layer representation from each view to a common consensus representation. Meanwhile, an efficient algorithm using alternating multiplicative update rules is developed. Furthermore, in order to demonstrate the effectiveness of this proposed method, we employ several open datasets including image and text datasets to evaluate the clustering performance of MvDGNMF and the state-of-art methods.

[1]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Hal Daumé,et al.  A Co-training Approach for Multi-view Spectral Clustering , 2011, ICML.

[3]  Jianhai Zhang,et al.  Deep Multimodal Multilinear Fusion with High-order Polynomial Pooling , 2019, NeurIPS.

[4]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[5]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[6]  George Trigeorgis,et al.  A Deep Matrix Factorization Method for Learning Attribute Representations , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[8]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[9]  Rafal Zdunek,et al.  Segmented convex-hull algorithms for near-separable NMF and NTF , 2019, Neurocomputing.

[10]  Chris H. Q. Ding,et al.  K-means clustering via principal component analysis , 2004, ICML.

[11]  Sang Min Yoon,et al.  Subspace clustering via structure-enforced dictionary learning , 2019, Neurocomputing.

[12]  Jun Yu,et al.  Local Deep-Feature Alignment for Unsupervised Dimension Reduction , 2018, IEEE Transactions on Image Processing.

[13]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[14]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[15]  Hong Yu,et al.  Self-Weighted Multi-View Clustering with Deep Matrix Factorization , 2019, ACML.

[16]  A. Cichocki,et al.  Multilayer nonnegative matrix factorisation , 2006 .

[17]  Sham M. Kakade,et al.  Multi-view clustering via canonical correlation analysis , 2009, ICML '09.

[18]  Hong Yu,et al.  Multi-view clustering via multi-manifold regularized non-negative matrix factorization , 2017, Neural Networks.

[19]  Xue Li,et al.  Graph regularized multilayer concept factorization for data representation , 2017, Neurocomputing.

[20]  Yun Fu,et al.  Multi-View Clustering via Deep Matrix Factorization , 2017, AAAI.

[21]  Shiliang Sun,et al.  Multi-view learning overview: Recent progress and new challenges , 2017, Inf. Fusion.

[22]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[23]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[24]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[25]  Jian Yang,et al.  Robust unsupervised feature selection by nonnegative sparse subspace learning , 2019, Neurocomputing.

[26]  Ruiqiang He,et al.  Discriminative and coherent subspace clustering , 2018, Neurocomputing.

[27]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[28]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[29]  Derek Greene,et al.  A Matrix Factorization Approach for Integrating Multiple Data Views , 2009, ECML/PKDD.

[30]  Haojie Li,et al.  Sparse dual graph-regularized NMF for image co-clustering , 2018, Neurocomputing.

[31]  Ming Li,et al.  Feature extraction via multi-view non-negative matrix factorization with local graph regularization , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[32]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[33]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[34]  Meng Wang,et al.  Multimodal Deep Autoencoder for Human Pose Recovery , 2015, IEEE Transactions on Image Processing.

[35]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[36]  Qingming Huang,et al.  Spatial Pyramid-Enhanced NetVLAD With Weighted Triplet Loss for Place Recognition , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Jun Yu,et al.  Hierarchical Deep Click Feature Prediction for Fine-Grained Image Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Feiping Nie,et al.  Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Multi-View K-Means Clustering on Big Data , 2022 .

[39]  Hong Yu,et al.  Constrained NMF-Based Multi-View Clustering on Unmapped Data , 2015, AAAI.

[40]  I T Joliffe,et al.  Principal component analysis and exploratory factor analysis , 1992, Statistical methods in medical research.

[41]  Louis-Philippe Morency,et al.  Multimodal Machine Learning: A Survey and Taxonomy , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Jiawei Han,et al.  Multi-View Clustering via Joint Nonnegative Matrix Factorization , 2013, SDM.

[43]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[44]  Stéphane Marchand-Maillet,et al.  Multiview clustering: a late fusion approach using latent models , 2009, SIGIR.

[45]  Hao Wang,et al.  Multi-view Clustering via Concept Factorization with Local Manifold Regularization , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).