Deep Generative Models for Relational Data with Side Information

We present a probabilistic framework for overlapping community discovery and link prediction for relational data, given as a graph. The proposed framework has: (1) a deep architecture which enables us to infer multiple layers of latent features/communities for each node, providing superior link prediction performance on more complex networks and better interpretability of the latent features; and (2) a regression model which allows directly conditioning the node latent features on the side information available in form of node attributes. Our framework handles both (1) and (2) via a clean, unified model, which enjoys full local conjugacy via data augmentation, and facilitates efficient inference via closed form Gibbs sampling. Moreover, inference cost scales in the number of edges which is attractive for massive but sparse networks. Our framework is also easily extendable to model weighted networks with count-valued edges. We compare with various state-of-the-art methods and report results, both quantitative and qualitative, on several benchmark data sets.

[1]  Aaron Clauset,et al.  Adapting the Stochastic Block Model to Edge-Weighted Networks , 2013, ArXiv.

[2]  Faten Ghosn,et al.  The MID3 Data Set, 1993—2001: Procedures, Coding Rules, and Description , 2004 .

[3]  Edoardo M. Airoldi,et al.  A Survey of Statistical Network Models , 2009, Found. Trends Mach. Learn..

[4]  Jure Leskovec,et al.  Community-Affiliation Graph Model for Overlapping Network Community Detection , 2012, 2012 IEEE 12th International Conference on Data Mining.

[5]  Edoardo M. Airoldi,et al.  Mixed Membership Stochastic Blockmodels , 2007, NIPS.

[6]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[7]  Zoubin Ghahramani,et al.  An Infinite Latent Attribute Model for Network Data , 2012, ICML.

[8]  James G. Scott,et al.  Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables , 2012, 1205.0310.

[9]  T. Snijders,et al.  Estimation and Prediction for Stochastic Blockstructures , 2001 .

[10]  D. Dunson,et al.  Bayesian latent variable models for mixed discrete outcomes. , 2005, Biostatistics.

[11]  Yoshihiro Yamanishi,et al.  Supervised enzyme network inference from the integration of genomic data and chemical information , 2005, ISMB.

[12]  P. Latouche,et al.  Overlapping stochastic block models with application to the French political blogosphere , 2009, 0910.2098.

[13]  Radford M. Neal.  Connectionist Learning of Belief Networks , 1992, Artif. Intell..

[14]  Le Song,et al.  A Multiscale Community Blockmodel for Network Exploration , 2011, AISTATS.

[15]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[16]  Mingyuan Zhou,et al.  Infinite Edge Partition Models for Overlapping Community Detection and Link Prediction , 2015, AISTATS.

[17]  David M. Blei,et al.  Bayesian Nonparametric Poisson Factorization for Recommendation Systems , 2014, AISTATS.

[18]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[19]  Lawrence Carin,et al.  Topic-Based Embeddings for Learning from Large Knowledge Graphs , 2016, AISTATS.

[20]  Zhe Gan,et al.  Learning Deep Sigmoid Belief Networks with Data Augmentation , 2015, AISTATS.

[21]  Thomas L. Griffiths,et al.  Nonparametric Latent Feature Models for Link Prediction , 2009, NIPS.

[22]  Erik B. Sudderth,et al.  The Nonparametric Metadata Dependent Relational Model , 2012, ICML.

[23]  Peter D. Hoff,et al.  Latent Space Approaches to Social Network Analysis , 2002 .

[24]  Morten Mørup,et al.  Nonparametric Bayesian modeling of complex networks: an introduction , 2013, IEEE Signal Processing Magazine.

[25]  David B. Dunson,et al.  Beta-Negative Binomial Process and Poisson Factor Analysis , 2011, AISTATS.

[26]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[27]  Jun Zhu,et al.  Max-Margin Nonparametric Latent Feature Models for Link Prediction , 2012, ICML.

[28]  Charles Elkan,et al.  Link Prediction via Matrix Factorization , 2011, ECML/PKDD.

[29]  Jure Leskovec,et al.  Learning to Discover Social Circles in Ego Networks , 2012, NIPS.