Multi-scale graph attention subspace clustering network

Abstract Deep subspace clustering (DSC) network which uses deep auto-encoders (DAE) mapping raw data into a latent space for clustering has achieved significant performance. However, when it comes to graph-based clustering, it fails to encode the node attribute and graph structure simultaneously, which degrades the clustering performance. Meanwhile, existing graph convolutional network (GCN) based clustering methods usually do not have a clustering-oriented objection function since the process of learning representation and clustering are separated. In addition, both aforementioned methods can neither catch the multi-scale information nor use the current clustering labels effectively, which lead to the suboptimal performance. To this end, we proposed a novel end-to-end framework called Multi-Scale Graph Attention Subspace Clustering Network (MSGA). By employing a novel GCN-based feature extraction module, it can effectively capture the node representation on graph-based datasets. Moreover, a multi-scale self-expression module is designed for obtaining a more discriminative coefficient representation from each layer of the encoder and a self-supervised module is introduced for supervising the learning of node representation. Specifically, we train and optimize them in a unified framework so that different modules can benefit from each other. Extensive experiments demonstrate the superiority of our method compared with several state-of-the-art methods.

[1]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[2]  René Vidal,et al.  Structured Sparse Subspace Clustering: A unified optimization framework , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Chun Wang,et al.  MGAE: Marginalized Graph Autoencoder for Graph Clustering , 2017, CIKM.

[4]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[5]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[6]  Ruichu Cai,et al.  Block diagonal representation learning for robust subspace clustering , 2020, Inf. Sci..

[7]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[8]  Xinbo Gao,et al.  Multiview Subspace Clustering by an Enhanced Tensor Nuclear Norm , 2021, IEEE Transactions on Cybernetics.

[9]  Enhong Chen,et al.  Learning Deep Representations for Graph Clustering , 2014, AAAI.

[10]  Xinbo Gao,et al.  Self-supervised graph convolutional clustering by preserving latent distribution , 2021, Neurocomputing.

[11]  Xinbo Gao,et al.  Self-Supervised Graph Convolutional Network for Multi-View Clustering , 2021, IEEE Transactions on Multimedia.

[12]  Jing Jiang,et al.  Attributed Graph Clustering: A Deep Attentional Embedding Approach , 2019, IJCAI.

[13]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[14]  Vishal M. Patel,et al.  Deep Multimodal Subspace Clustering Networks , 2018, IEEE Journal of Selected Topics in Signal Processing.

[15]  René Vidal,et al.  Latent Space Sparse Subspace Clustering , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Honggang Zhang,et al.  Low-rank and structured sparse subspace clustering , 2016, 2016 Visual Communications and Image Processing (VCIP).

[17]  Guihua Tao,et al.  Exsavi: Excavating both sample-wise and view-wise relationships to boost multi-view subspace clustering , 2020, Neurocomputing.

[18]  Zhang Yi,et al.  Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering , 2012, IEEE Transactions on Cybernetics.

[19]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[20]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[21]  Satu Elisa Schaeffer,et al.  Graph Clustering , 2017, Encyclopedia of Machine Learning and Data Mining.

[22]  Bruce A. Draper,et al.  Semi-Nonnegative Matrix Factorization for Motion Segmentation with Missing Data , 2012, ECCV.

[23]  Jacek M. Zurada,et al.  Normalized Mutual Information Feature Selection , 2009, IEEE Transactions on Neural Networks.

[24]  Junbin Gao,et al.  Subspace Clustering via Learning an Adaptive Low-Rank Graph , 2018, IEEE Transactions on Image Processing.

[25]  Hongdong Li,et al.  Efficient dense subspace clustering , 2014, IEEE Winter Conference on Applications of Computer Vision.

[26]  Hong Peng,et al.  Deep subspace clustering to achieve jointly latent feature extraction and discriminative learning , 2020, Neurocomputing.

[27]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[28]  Junbin Gao,et al.  Kernel Sparse Subspace Clustering on Symmetric Positive Definite Manifolds , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Hasan Davulcu,et al.  Graph Attention Auto-Encoders , 2019, 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI).

[30]  J. Munkres ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[31]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[32]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[33]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[34]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[35]  Y. Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, CVPR 2004.

[36]  René Vidal,et al.  Low rank subspace clustering (LRSC) , 2014, Pattern Recognit. Lett..

[37]  Jiawei Han,et al.  Document clustering using locality preserving indexing , 2005, IEEE Transactions on Knowledge and Data Engineering.

[38]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[39]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[40]  Pu Zhang,et al.  Tensor-SVD Based Graph Learning for Multi-View Subspace Clustering , 2020, AAAI.

[41]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[42]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[43]  Daniel P. Robinson,et al.  Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[45]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  René Vidal,et al.  Kernel sparse subspace clustering , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[47]  Shengli Xie,et al.  Deep Clustering via Weighted $k$-Subspace Network , 2019, IEEE Signal Processing Letters.

[48]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[49]  Mark J. Embrechts,et al.  On the Use of the Adjusted Rand Index as a Metric for Evaluating Supervised Classification , 2009, ICANN.

[50]  Xiao Wang,et al.  Structural Deep Clustering Network , 2020, WWW.