A Block-based Generative Model for Attributed Networks Embedding

Attributed network embedding has attracted plenty of interests in recent years. It aims to learn task-independent, low-dimension, and continuous vectors for nodes preserving both topology and attribute information. Most existing methods, such as GCN and its variations, mainly focus on the local information, i.e., the attributes of the neighbors. Thus, they have been well studied for assortative networks but ignored disassortative networks, which are common in real scenes. To address this issue, we propose a block-based generative model for attributed network embedding on a probability perspective inspired by the stochastic block model (SBM). Specifically, the nodes are assigned to several blocks wherein the nodes in the same block share the similar link patterns. These patterns can define assortative networks containing communities or disassortative networks with the multipartite, hub, or any hybrid structures. Concerning the attribute information, we assume that each node has a hidden embedding related to its assigned block, and then we use a neural network to characterize the nonlinearity between the node embedding and its attribute. We perform extensive experiments on real-world and synthetic attributed networks, and the experimental results show that our proposed method remarkably outperforms state-of-the-art embedding methods for both clustering and classification tasks, especially on disassortative networks.

[1]  Huan Liu,et al.  Unsupervised Streaming Feature Selection in Social Media , 2015, CIKM.

[2]  David M. Blei,et al.  Scalable Recommendation with Hierarchical Poisson Factorization , 2015, UAI.

[3]  Laurens van der Maaten,et al.  Accelerating t-SNE using tree-based algorithms , 2014, J. Mach. Learn. Res..

[4]  Stephan Günnemann,et al.  Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking , 2017, ICLR.

[5]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[6]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[7]  Kai Zheng,et al.  Temporal paths discovery with multiple constraints in attributed dynamic graphs , 2019, World Wide Web.

[8]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[9]  Christophe Ambroise,et al.  Variational Bayesian inference and complexity control for stochastic block models , 2009, 0912.2873.

[10]  Dayou Liu,et al.  Characterizing and Extracting Multiplex Patterns in Complex Networks , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Fan Yang,et al.  Large-Scale Heterogeneous Feature Embedding , 2019, AAAI.

[12]  Lise Getoor,et al.  Query-driven Active Surveying for Collective Classification , 2012 .

[13]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[14]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[15]  Chengqi Zhang,et al.  Binarized attributed network embedding , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[16]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[17]  Ludmila I. Kuncheva,et al.  Using diversity in cluster ensembles , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[18]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[19]  Emmanuel Müller,et al.  Focused clustering and outlier detection in large attributed graphs , 2014, KDD.

[20]  Nicola Barbieri,et al.  Who to follow and why: link prediction with explanations , 2014, KDD.

[21]  Heng Huang,et al.  Deep Attributed Network Embedding , 2018, IJCAI.

[22]  Yanchun Zhang,et al.  Community Detection in Attributed Graphs: An Embedding Approach , 2018, AAAI.

[23]  Lu Chen,et al.  Contextual Community Search Over Large Social Networks , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[24]  Kevin Chen-Chuan Chang,et al.  Geom-GCN: Geometric Graph Convolutional Networks , 2020, ICLR.

[25]  Yihong Gong,et al.  Detecting communities and their evolutions in dynamic social networks—a Bayesian approach , 2011, Machine Learning.

[26]  Weiqing Wang,et al.  Social Boosted Recommendation With Folded Bipartite Network Embedding , 2020, IEEE Transactions on Knowledge and Data Engineering.

[27]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[28]  P. Holland,et al.  An Exponential Family of Probability Distributions for Directed Graphs , 1981 .

[29]  Philip S. Yu,et al.  On Spectral Graph Embedding: A Non-Backtracking Perspective and Graph Approximation , 2018, SDM.

[30]  Jonathan Qiang Jiang,et al.  Stochastic Blockmodel and Exploratory Analysis in Signed Networks , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[31]  Katarzyna Musial,et al.  How to predict social relationships — Physics-inspired approach to link prediction , 2019, Physica A: Statistical Mechanics and its Applications.

[32]  Bo Yang,et al.  Stochastic Blockmodeling and Variational Bayes Learning for Signed Network Analysis , 2017, IEEE Transactions on Knowledge and Data Engineering.

[33]  Katarzyna Musial,et al.  Change point detection in social networks - Critical review with experiments , 2018, Comput. Sci. Rev..

[34]  Shuiwang Ji,et al.  Graph U-Nets , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[36]  Hasan Davulcu,et al.  Graph Attention Auto-Encoders , 2019, 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI).

[37]  Roger Guimerà,et al.  Missing and spurious interactions and the reconstruction of complex networks , 2009, Proceedings of the National Academy of Sciences.

[38]  Ash A. Alizadeh,et al.  Corrigendum: Circulating tumour DNA profiling reveals heterogeneity of EGFR inhibitor resistance mechanisms in lung cancer patients , 2016, Nature Communications.

[39]  Lawrence Carin,et al.  Stochastic Blockmodels meet Graph Neural Networks , 2019, ICML.

[40]  Ming Gao,et al.  BiNE: Bipartite Network Embedding , 2018, SIGIR.

[41]  Xiangnan He,et al.  Attributed Social Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[42]  Wen-Chih Peng,et al.  Exploiting Centrality Information with Graph Convolutions for Network Representation Learning , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[43]  Mark E. J. Newman,et al.  Structure and inference in annotated networks , 2015, Nature Communications.

[44]  Fabio Roli,et al.  F-measure optimisation in multi-label classifiers , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[45]  Srinivasan Parthasarathy,et al.  Efficient community detection in large networks using content and links , 2012, WWW.

[46]  Alejandro Ribeiro,et al.  Diffusion Scattering Transforms on Graphs , 2018, ICLR.

[47]  Yizhou Sun,et al.  On community outliers and their efficient detection in information networks , 2010, KDD.

[48]  Alberto H. F. Laender,et al.  Social-Based Classification of Multiple Interactions in Dynamic Attributed Networks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[49]  Tom M. Mitchell,et al.  Learning to Extract Symbolic Knowledge from the World Wide Web , 1998, AAAI/IAAI.

[50]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[51]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[52]  Emmanuel Abbe,et al.  Community detection and stochastic block models: recent developments , 2017, Found. Trends Commun. Inf. Theory.

[53]  Xueqi Cheng,et al.  Graph Wavelet Neural Network , 2019, ICLR.

[54]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[55]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[56]  Jiajun Bu,et al.  ANRL: Attributed Network Representation Learning via Deep Neural Networks , 2018, IJCAI.

[57]  Rui Zhou,et al.  Finding Attribute Diversified Communities in Complex Networks , 2020, DASFAA.

[58]  Mohamed R. Amer,et al.  Understanding Attention and Generalization in Graph Neural Networks , 2019, NeurIPS.

[59]  Bo Yang,et al.  Enhanced Network Embedding with Text Information , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[60]  Yuchen Li,et al.  A survey of typical attributed graph queries , 2020, World Wide Web.

[61]  Xiao Huang,et al.  Accelerated Attributed Network Embedding , 2017, SDM.

[62]  Young-Koo Lee,et al.  Set-based unified approach for summarization of a multi-attributed graph , 2017, World Wide Web.