Optimizing Variational Graph Autoencoder for Community Detection

Variational Graph Autoencoders (VGAE) has recently been a popular framework of choice for learning representations on graphs. Its inception has allowed models to achieve state-of-the-art performances for challenging tasks such as link prediction, rating prediction and node clustering. However, a fundamental flaw exists in Variational Autoencoder (VAE) based approaches. Specifically, the objective function of VAE (reconstruction loss), deviates from its primary objective (i.e clustering). In this paper, we attempt to address this issue by introducing two significant changes to Variational Graph Autoencoder for Community Detection (VGAECD). Firstly, we introduce a simplified graph convolution encoder to increase convergence speed and reduce computational time. Secondly, a dual variational objective is introduced to encourage learning of the primary objective. The outcome is a faster converging model with competitive community detection performance.

[1]  Enhong Chen,et al.  Learning Deep Representations for Graph Clustering , 2014, AAAI.

[2]  Tsuyoshi Murata,et al.  A General View for Network Embedding as Matrix Factorization , 2019, WSDM.

[3]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[4]  Leon Danon,et al.  Comparing community structure identification , 2005, cond-mat/0505245.

[5]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[6]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[7]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[8]  T. Snijders,et al.  Estimation and Prediction for Stochastic Blockmodels for Graphs with Latent Block Structure , 1997 .

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[11]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[12]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[14]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[15]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[16]  Richard M. Karp,et al.  Algorithms for graph partitioning on the planted partition model , 2001, Random Struct. Algorithms.

[17]  Richard M. Karp,et al.  Algorithms for graph partitioning on the planted partition model , 1999, Random Struct. Algorithms.

[18]  Tsuyoshi Murata,et al.  Learning Community Structure with Variational Autoencoder , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[19]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[20]  Chun Wang,et al.  MGAE: Marginalized Graph Autoencoder for Graph Clustering , 2017, CIKM.

[21]  J. Pearl.  Causal inference in statistics: An overview , 2009 .

[22]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[24]  F. Radicchi,et al.  Benchmark graphs for testing community detection algorithms. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Mark E. J. Newman,et al.  Stochastic blockmodels and community structure in networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[26]  Le Song,et al.  Coupled Variational Bayes via Optimization Embedding , 2018, NeurIPS.

[27]  Jascha Sohl-Dickstein,et al.  Meta-Learning Update Rules for Unsupervised Representation Learning , 2018, ICLR.

[28]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[29]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[30]  Santosh S. Vempala,et al.  On clusterings-good, bad and spectral , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[31]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[32]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[33]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[34]  Jure Leskovec,et al.  Defining and evaluating network communities based on ground-truth , 2012, Knowledge and Information Systems.

[35]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[36]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[37]  Jürgen Schmidhuber,et al.  Neural Expectation Maximization , 2017, NIPS.

[38]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[39]  Santo Fortunato,et al.  Community detection in networks: A user guide , 2016, ArXiv.

[40]  M E J Newman,et al.  Finding and evaluating community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.