Gravity-Inspired Graph Autoencoders for Directed Link Prediction

Graph autoencoders (AE) and variational autoencoders (VAE) recently emerged as powerful node embedding methods. In particular, graph AE and VAE were successfully leveraged to tackle the challenging link prediction problem, aiming at figuring out whether some pairs of nodes from a graph are connected by unobserved edges. However, these models focus on undirected graphs and therefore ignore the potential direction of the link, which is limiting for numerous real-life applications. In this paper, we extend the graph AE and VAE frameworks to address link prediction in directed graphs. We present a new gravity-inspired decoder scheme that can effectively reconstruct directed graphs from a node embedding. We empirically evaluate our method on three different directed link prediction tasks, for which standard graph AE and VAE perform poorly. We achieve competitive results on three real-world graphs, outperforming several popular baselines.

[1]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[2]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[3]  Michalis Vazirgiannis,et al.  A Degeneracy Framework for Scalable Graph Autoencoders , 2019, IJCAI.

[4]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[5]  Yan Yu,et al.  Link Prediction in Directed Network and Its Application in Microblog , 2015 .

[6]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[7]  Stefano Ermon,et al.  Graphite: Iterative Generative Modeling of Graphs , 2018, ICML.

[8]  Ulrik Brandes,et al.  On variants of shortest-path betweenness centrality and their generic computation , 2008, Soc. Networks.

[9]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[10]  Phi Vu Tran,et al.  Learning to Make Predictions on Graphs with Autoencoders , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[11]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[12]  Fan Meng,et al.  Recommendation Algorithm based on Link Prediction and Domain Knowledge in Retail Transactions , 2014, ITQM.

[13]  Henry Cavendish,et al.  Experiments to determine the Density of the Earth , 2010 .

[14]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[15]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Albert-László Barabási,et al.  Network-based prediction of protein interactions , 2018, Nature Communications.

[17]  Dimitrios M. Thilikos,et al.  D-cores: measuring collaboration of directed graphs based on degeneracy , 2011, Knowledge and Information Systems.

[18]  David Eppstein,et al.  Force-Directed Graph Drawing Using Social Gravity and Scaling , 2012, GD.

[19]  Wilfried Philips,et al.  Matrix Completion with Variational Graph Autoencoders: Application in Hyperlocal Air Quality Inference , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Michalis Vazirgiannis,et al.  The core decomposition of networks: theory, algorithms and applications , 2019, The VLDB Journal.

[21]  David Liben-Nowell,et al.  The link-prediction problem for social networks , 2007 .

[22]  Chengqi Zhang,et al.  Network Representation Learning: A Survey , 2017, IEEE Transactions on Big Data.

[23]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Niloy Ganguly,et al.  NeVAE: A Deep Generative Model for Molecular Graphs , 2018, AAAI.

[26]  Jon M. Kleinberg,et al.  The link-prediction problem for social networks , 2007, J. Assoc. Inf. Sci. Technol..

[27]  Qi Liu,et al.  Constrained Graph Variational Autoencoders for Molecule Design , 2018, NeurIPS.

[28]  Chun Wang,et al.  MGAE: Marginalized Graph Autoencoder for Graph Clustering , 2017, CIKM.

[29]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[30]  Mohammad Reza Keyvanpour,et al.  A systemic analysis of link prediction in social network , 2019, Artificial Intelligence Review.

[31]  Cao Xiao,et al.  Constrained Generation of Semantically Valid Graphs via Regularizing Variational Autoencoders , 2018, NeurIPS.

[32]  Éva Tardos,et al.  Maximizing the Spread of Influence through a Social Network , 2015, Theory Comput..

[33]  Olivier Bachem,et al.  Recent Advances in Autoencoder-Based Representation Learning , 2018, ArXiv.

[34]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[35]  I. Newton.  Philosophiæ naturalis principia mathematica , 1973 .

[36]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[37]  Chang Zhou,et al.  Scalable Graph Embedding for Asymmetric Proximity , 2017, AAAI.

[38]  Regina Barzilay,et al.  Junction Tree Variational Autoencoder for Molecular Graph Generation , 2018, ICML.

[39]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[40]  Albert Einstein,et al.  Erklärung der Perihelbewegung des Merkur aus der allgemeinen Relativitätstheorie , 2006 .

[41]  Cao Xiao,et al.  FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling , 2018, ICLR.

[42]  Dario Garcia Gasulla.  Link prediction in large directed graphs , 2015 .

[43]  Pierre Baldi,et al.  Autoencoders, Unsupervised Learning, and Deep Architectures , 2011, ICML Unsupervised and Transfer Learning.

[44]  Daniel Schall,et al.  Link Prediction for Directed Graphs , 2015 .

[45]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[46]  Edward M. Reingold,et al.  Graph drawing by force‐directed placement , 1991, Softw. Pract. Exp..

[47]  Max Welling,et al.  Graph Convolutional Matrix Completion , 2017, ArXiv.

[48]  Peng Wang,et al.  Link prediction in social networks: the state-of-the-art , 2014, Science China Information Sciences.

[49]  Roman Garnett,et al.  D-VAE: A Variational Autoencoder for Directed Acyclic Graphs , 2019, NeurIPS.

[50]  Lingling Zhang,et al.  Genre-based Link Prediction in Bipartite Graph for Music Recommendation☆ , 2016 .

[51]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[52]  Nikos Komodakis,et al.  GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders , 2018, ICANN.

[53]  Katarzyna Musial,et al.  Newton's Gravitational Law for Link Prediction in Social Networks , 2017, COMPLEX NETWORKS.

[54]  Thomas L. Griffiths,et al.  Nonparametric Latent Feature Models for Link Prediction , 2009, NIPS.

[55]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, ArXiv.

[56]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[57]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[58]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[59]  Leo Katz,et al.  A new status index derived from sociometric analysis , 1953 .

[60]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[61]  Michalis Vazirgiannis,et al.  Clustering and Community Detection in Directed Networks: A Survey , 2013, ArXiv.

[62]  Henry Cavendish,et al.  XXI. Experiments to determine the density of the earth , 2022, Philosophical Transactions of the Royal Society of London.