Graph Clustering via Variational Graph Embedding

Abstract Graph clustering based on embedding aims to divide nodes with higher similarity into several mutually disjoint groups, but it is not a trivial task to maximumly embed the graph structure and node attributes into the low dimensional feature space. Furthermore, most of the current advanced methods of graph nodes clustering adopt the strategy of separating graph embedding technology and clustering algorithm, and ignore the potential relationship between them. Therefore, we propose an innovative end-to-end graph clustering framework with joint strategy to handle the complex problem in a non-Euclidean space. In terms of learning the graph embedding, we propose a new variational graph auto-encoder algorithm based on the Graph Convolution Network (GCN), which takes into account the boosting influence of joint generative model of graph structure and node attributes on the embedding output. On the basis of embedding representation, we implement a self-training mechanism through the construction of auxiliary distribution to further enhance the prediction of node categories, thereby realizing the unsupervised clustering mode. In addition, the loss contribution of each cluster is normalized to prevent large clusters from distorting the embedding space. Extensive experiments on real-world graph datasets validate our design and demonstrate that our algorithm has highly competitive in graph clustering over state-of-the-art methods.

[1]  Meng Wang,et al.  Learning content–social influential features for influence analysis , 2016, International Journal of Multimedia Information Retrieval.

[2]  Chang-Dong Wang,et al.  Discriminative metric learning for multi-view graph partitioning , 2018, Pattern Recognit..

[3]  Ali Farhadi,et al.  Unsupervised Deep Embedding for Clustering Analysis , 2015, ICML.

[4]  Philip S. Yu,et al.  A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning , 2021, IEEE Transactions on Knowledge and Data Engineering.

[5]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[6]  Luc Devroye,et al.  Sample-based non-uniform random variate generation , 1986, WSC '86.

[7]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[8]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[9]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[10]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[11]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[12]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[13]  Zenglin Xu,et al.  Structured Graph Learning for Clustering and Semi-supervised Classification , 2020, Pattern Recognit..

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Fei Wang,et al.  Graph dual regularization non-negative matrix factorization for co-clustering , 2012, Pattern Recognit..

[16]  Reda Alhajj,et al.  CutESC: Cutting edge spatial clustering technique based on proximity graphs , 2019, Pattern Recognit..

[17]  Lei Du,et al.  Robust Multi-View Spectral Clustering via Low-Rank and Sparse Decomposition , 2014, AAAI.

[18]  David M. Blei,et al.  Relational Topic Models for Document Networks , 2009, AISTATS.

[19]  Enhong Chen,et al.  Learning Deep Representations for Graph Clustering , 2014, AAAI.

[20]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[21]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[22]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[23]  James P. Callan,et al.  Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding , 2017, WWW.

[24]  Meixia Qu,et al.  A Deep Graph Structured Clustering Network , 2020, IEEE Access.

[25]  Tong Wu,et al.  Graph regularized low-rank representation for submodule clustering , 2020, Pattern Recognit..

[26]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[27]  Zenglin Xu,et al.  Auto-weighted multi-view clustering via kernelized graph learning , 2019, Pattern Recognit..

[28]  Alexander J. Smola,et al.  Distributed large-scale natural graph factorization , 2013, WWW.

[29]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[30]  Xiaoning Qian,et al.  Semi-Implicit Graph Variational Auto-Encoders , 2019, NeurIPS.

[31]  Hong Cheng,et al.  Dirichlet Graph Variational Autoencoder , 2020, NeurIPS.

[32]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[33]  Tsuyoshi Murata,et al.  Variational Approach for Learning Community Structures , 2018, Complex..

[34]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[35]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[36]  Hasan Davulcu,et al.  Graph Attention Auto-Encoders , 2019, 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI).

[37]  Chun Wang,et al.  MGAE: Marginalized Graph Autoencoder for Graph Clustering , 2017, CIKM.

[38]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.