ANAE: Learning Node Context Representation for Attributed Network Embedding

Attributed network embedding aims to learn low-dimensional node representations from both network structure and node attributes. Existing methods can be categorized into two groups: (1) the first group learns two separated node representations from network structure and node attribute respectively and concatenates them together; (2) the other group obtains node representations by translating node attributes into network structure or vice versa. However, both groups have their drawbacks. The first group neglects the correlation between network structure and node attributes, while the second group assumes strong dependence between these two types of information. In this paper, we address attributed network embedding from a novel perspective, i.e., learning node context representation for each node via modeling its attributed local subgraph. To achieve this goal, we propose a novel attributed network auto-encoder framework, namely ANAE. For a target node, ANAE first aggregates the attribute information from its attributed local subgraph, obtaining its low-dimensional representation. Next, ANAE diffuses the representation of the target node to nodes in its local subgraph to reconstruct their attributes. Such an encoder-decoder framework allows the learned representations to better preserve the context information manifested in both network structure and node attributes, thus having high capacity to learn good node representations for attributed network. Extensive experimental results on real-world datasets demonstrate that the proposed framework outperforms the state-of-the-art approaches at the tasks of link prediction and node classification.

[1]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[2]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[3]  M. Narasimha Murty,et al.  Outlier Aware Network Embedding for Attributed Networks , 2018, AAAI.

[4]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[5]  Zhiyuan Liu,et al.  Max-Margin DeepWalk: Discriminative Learning of Network Representation , 2016, IJCAI.

[6]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[7]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[8]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[9]  Huawei Shen,et al.  Graph Convolutional Networks using Heat Kernel for Semi-supervised Learning , 2019, IJCAI.

[10]  Danai Koutra,et al.  RolX: structural role extraction & mining in large graphs , 2012, KDD.

[11]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[12]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Xiaolong Jin,et al.  Predict Anchor Links across Social Networks via an Embedding Approach , 2016, IJCAI.

[15]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[16]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[17]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[18]  Jiajun Bu,et al.  ANRL: Attributed Network Representation Learning via Deep Neural Networks , 2018, IJCAI.

[19]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[20]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[21]  Chengqi Zhang,et al.  Binarized attributed network embedding , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[22]  Minyi Guo,et al.  GraphGAN: Graph Representation Learning with Generative Adversarial Nets , 2017, AAAI.

[23]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[25]  Heng Huang,et al.  Deep Attributed Network Embedding , 2018, IJCAI.

[26]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[27]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[28]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[29]  Huan Liu,et al.  Attributed Network Embedding for Learning in a Dynamic Environment , 2017, CIKM.

[30]  Yihong Gong,et al.  Combining content and link for classification using matrix factorization , 2007, SIGIR.

[31]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[32]  Yang Yang,et al.  Representation Learning for Scale-free Networks , 2017, AAAI.

[33]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[34]  Hady Wirawan Lauw,et al.  Probabilistic Latent Document Network Embedding , 2014, 2014 IEEE International Conference on Data Mining.

[35]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[36]  Heng Ji,et al.  Exploring Context and Content Links in Social Media: A Latent Space Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Pietro Liò,et al.  Deep Graph Infomax , 2018, ICLR.

[38]  Xiangnan He,et al.  Attributed Social Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[39]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[40]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[41]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[42]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[43]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[44]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[45]  Xiao Huang,et al.  Accelerated Attributed Network Embedding , 2017, SDM.

[46]  Xiao Huang,et al.  Label Informed Attributed Network Embedding , 2017, WSDM.

[47]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[48]  Zhaochun Ren,et al.  Multi-Dimensional Network Embedding with Hierarchical Structure , 2018, WSDM.

[49]  Le Song,et al.  Stochastic Training of Graph Convolutional Networks with Variance Reduction , 2017, ICML.

[50]  Xiao Huang,et al.  Exploring Expert Cognition for Attributed Network Embedding , 2018, WSDM.

[51]  Hari Sundaram,et al.  Growing Attributed Networks through Local Processes , 2017, WWW.

[52]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[53]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[54]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[55]  Ludovic Denoyer,et al.  Learning latent representations of nodes for classifying in heterogeneous social networks , 2014, WSDM.

[56]  Cao Xiao,et al.  FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling , 2018, ICLR.

[57]  Xueqi Cheng,et al.  Graph Wavelet Neural Network , 2019, ICLR.

[58]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[59]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.