Deep neighbor-aware embedding for node clustering in attributed graphs

Abstract Node clustering aims to partition the vertices in a graph into multiple groups or communities. Existing studies have mostly focused on developing deep learning approaches to learn a latent representation of nodes, based on which simple clustering methods like k -means are applied. These two-step frameworks for node clustering are difficult to manipulate and usually lead to suboptimal performance, mainly because the graph embedding is not goal-directed, i.e., designed for the specific clustering task. In this paper, we propose a clustering-directed deep learning approach, Deep Neighbor-aware Embedded Node Clustering ( DNENC for short) for clustering graph data. Our method focuses on attributed graphs to sufficiently explore the two sides of information in graphs. It encodes the topological structure and node content in a graph into a compact representation via a neighbor-aware graph autoencoder, which progressively absorbs information from neighbors via a convolutional or attentional encoder. Multiple neighbor-aware encoders are stacked to build a deep architecture followed by an inner-product decoder for reconstructing the graph structure. Furthermore, soft labels are generated to supervise a self-training process, which iteratively refines the node clustering results. The self-training process is jointly learned and optimized with the graph embedding in a unified framework, to benefit both components mutually. Experimental results compared with state-of-the-art algorithms demonstrate the good performance of our framework.

[1]  Stefan Roth,et al.  Shrinkage Fields for Effective Image Restoration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[3]  Chengqi Zhang,et al.  CFOND: Consensus Factorization for Co-Clustering Networked Data , 2019, IEEE Transactions on Knowledge and Data Engineering.

[4]  Keith C. C. Chan,et al.  Deep Graph Clustering in Social Network , 2017, WWW.

[5]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[6]  Yizhou Sun,et al.  iTopicModel: Information Network-Integrated Topic Modeling , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[7]  Chun Wang,et al.  MGAE: Marginalized Graph Autoencoder for Graph Clustering , 2017, CIKM.

[8]  David M. Blei,et al.  Relational Topic Models for Document Networks , 2009, AISTATS.

[9]  Weiwei Liu,et al.  Discrete Network Embedding , 2018, IJCAI.

[10]  En Zhu,et al.  Deep Clustering with Convolutional Autoencoders , 2017, ICONIP.

[11]  Enhong Chen,et al.  Learning Deep Representations for Graph Clustering , 2014, AAAI.

[12]  Hong Yang,et al.  Active Discriminative Network Representation Learning , 2018, IJCAI.

[13]  Quanquan Gu,et al.  Co-clustering on manifolds , 2009, KDD.

[14]  M. Newman,et al.  Finding community structure in networks using the eigenvectors of matrices. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  M. Hastings Community detection as an inference problem. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  Chengqi Zhang,et al.  Co-clustering enterprise social networks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[17]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[18]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[19]  Cheng Deng,et al.  Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Su-Yeon Kim,et al.  Customer segmentation and strategy development based on customer lifetime value: A case study , 2006, Expert Syst. Appl..

[21]  Xiaotong Zhang,et al.  Attributed Graph Clustering via Adaptive Graph Convolution , 2019, IJCAI.

[22]  Laurens van der Maaten,et al.  Accelerating t-SNE using tree-based algorithms , 2014, J. Mach. Learn. Res..

[23]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Yu Xie,et al.  Community discovery in networks with deep sparse filtering , 2018, Pattern Recognit..

[25]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[26]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[27]  Philip S. Yu,et al.  A Survey on Knowledge Graphs: Representation, Acquisition, and Applications , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[28]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Pin-Yu Chen,et al.  Revisiting Spectral Graph Clustering with Generative Community Models , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[30]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[31]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[33]  Gordon Wetzstein,et al.  Unrolled Optimization with Deep Priors , 2017, ArXiv.

[34]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[35]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[36]  Jianping Yin,et al.  Improved Deep Embedded Clustering with Local Structure Preservation , 2017, IJCAI.

[37]  Randy C. Paffenroth,et al.  Anomaly Detection with Robust Deep Autoencoders , 2017, KDD.

[38]  Zhen Wang,et al.  Community Detection Based on Structure and Content: A Content Propagation Perspective , 2015, 2015 IEEE International Conference on Data Mining.

[39]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[40]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[41]  Lei Du,et al.  Robust Multi-View Spectral Clustering via Low-Rank and Sparse Decomposition , 2014, AAAI.

[42]  Yanchun Zhang,et al.  Community Detection in Attributed Graphs: An Embedding Approach , 2018, AAAI.

[43]  Jing Cheng,et al.  Deep MRI Reconstruction: Unrolled Optimization Algorithms Meet Neural Networks , 2019, ArXiv.

[44]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[45]  Mohammad-Reza Feizi-Derakhshi,et al.  Overlapping community detection in rating-based social networks through analyzing topics, ratings and links , 2018, Pattern Recognit..

[46]  Ali Farhadi,et al.  Unsupervised Deep Embedding for Clustering Analysis , 2015, ICML.

[47]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[48]  David A. Cohn,et al.  The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity , 2000, NIPS.

[49]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[50]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[51]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[52]  Stephan Günnemann,et al.  Bayesian Robust Attributed Graph Clustering: Joint Learning of Partial Anomalies and Group Structure , 2018, AAAI.