Graph Recurrent Networks With Attributed Random Walks

Random walks are widely adopted in various network analysis tasks ranging from network embedding to label propagation. It could capture and convert geometric structures into structured sequences while alleviating the issues of sparsity and curse of dimensionality. Though random walks on plain networks have been intensively studied, in real-world systems, nodes are often not pure vertices, but own different characteristics, described by the rich set of data associated with them. These node attributes contain plentiful information that often complements the network, and bring opportunities to the random-walk-based analysis. However, it is unclear how random walks could be developed for attributed networks towards an effective joint information extraction. Node attributes make the node interactions more complicated and are heterogeneous with respect to topological structures. To bridge the gap, we explore to perform joint random walks on attributed networks, and utilize them to boost the deep node representation learning. The proposed framework GraphRNA consists of two major components, i.e., a collaborative walking mechanism - AttriWalk, and a tailored deep embedding architecture for random walks, named graph recurrent networks (GRN). AttriWalk considers node attributes as a bipartite network and uses it to propel the walking more diverse and mitigate the tendency of converging to nodes with high centralities. AttriWalk enables us to advance the prominent deep network embedding model, graph convolutional networks, towards a more effective architecture - GRN. GRN empowers node representations to interact in the same way as nodes interact in the original attributed network. Experimental results on real-world datasets demonstrate the effectiveness of GraphRNA compared with the state-of-the-art embedding algorithms.

[1]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[2]  Jie Tang,et al.  ArnetMiner: extraction and mining of academic social networks , 2008, KDD.

[3]  Chang-Dong Wang,et al.  Attributed Network Embedding with Micro-Meso Structure , 2018, DASFAA.

[4]  Jure Leskovec,et al.  Inferring Networks of Substitutable and Complementary Products , 2015, KDD.

[5]  Jianbo Shi,et al.  A Random Walks View of Spectral Segmentation , 2001, AISTATS.

[6]  Cao Xiao,et al.  FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling , 2018, ICLR.

[7]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[8]  George Karypis,et al.  RecWalk: Nearly Uncoupled Random Walks for Top-N Recommendation , 2019, WSDM.

[9]  Heng Huang,et al.  Deep Attributed Network Embedding , 2018, IJCAI.

[10]  Colin Cooper,et al.  Random walks in recommender systems: exact computation and simulations , 2014, WWW.

[11]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[12]  Xiao Huang,et al.  Exploring Expert Cognition for Attributed Network Embedding , 2018, WSDM.

[13]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[14]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[15]  Xiao Huang,et al.  Accelerated Attributed Network Embedding , 2017, SDM.

[16]  Joseph JáJá,et al.  Learning Graph-Level Representations with Gated Recurrent Neural Networks , 2018, ArXiv.

[17]  Jure Leskovec,et al.  Supervised random walks: predicting and recommending links in social networks , 2010, WSDM '11.

[18]  Xiangnan He,et al.  Attributed Social Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[19]  Srinivasan Parthasarathy,et al.  SEANO: Semi-supervised Embedding in Attributed Networks with Outliers , 2017, SDM.

[20]  Luc Devroye,et al.  Sample-based non-uniform random variate generation , 1986, WSC '86.

[21]  Jiajun Bu,et al.  ANRL: Attributed Network Representation Learning via Deep Neural Networks , 2018, IJCAI.

[22]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[23]  Hady Wirawan Lauw,et al.  Probabilistic Latent Document Network Embedding , 2014, 2014 IEEE International Conference on Data Mining.

[24]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[25]  Aynaz Taheri,et al.  Learning Graph Representations with Recurrent Neural Network Autoencoders , 2018 .

[26]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[27]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[28]  Chengqi Zhang,et al.  Binarized attributed network embedding , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[29]  Andrew N. Smith,et al.  How Does Brand-related User-generated Content Differ across YouTube, Facebook, and Twitter? , 2012 .

[30]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[31]  Heng Ji,et al.  Exploring Context and Content Links in Social Media: A Latent Space Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[33]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[34]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[35]  Mason A. Porter,et al.  Random walks and diffusion on networks , 2016, ArXiv.

[36]  Xiao Huang,et al.  Accelerated Local Anomaly Detection via Resolving Attributed Networks , 2017, IJCAI.

[37]  Xiao Huang,et al.  Label Informed Attributed Network Embedding , 2017, WSDM.

[38]  Xiao Huang,et al.  A General Embedding Framework for Heterogeneous Information Learning in Large-Scale Networks , 2018, ACM Trans. Knowl. Discov. Data.

[39]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[40]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[41]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[42]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[43]  Huan Liu,et al.  Attributed Network Embedding for Learning in a Dynamic Environment , 2017, CIKM.

[44]  Yihong Gong,et al.  Combining content and link for classification using matrix factorization , 2007, SIGIR.

[45]  Sunil Prabhakar,et al.  A3embed: Attribute Association Aware Network Embedding , 2018, WWW.

[46]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.