ANE: Network Embedding via Adversarial Autoencoders

Network embedding is an important method to learn low-dimensional representations of vertexes in network, whose goal is to capture and preserve the highly non-linear network structures. Here, we propose an Adversarial autoencoders based Network Embedding method (ANE for short), which utilizes the rencently proposed adversarial autoencoders to perform variational inference by matching the aggregated posterior of low-dimensional representations of vertexes with an arbitraray prior distribution. This framework introduces adversarial regularization to autoencoders. And it is able to attaches the latent representations of similar vertexes to each other and thus prevents the manifold fracturing problem that is typically encountered in the embeddings learnt by the autoencoders. Experiments demonstrate the effictiveness of ANE on link prediction and multi-label classification on three real-world information networks.

[1]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[2]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[3]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[4]  David Liben-Nowell,et al.  The link-prediction problem for social networks , 2007 .

[5]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[6]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[7]  Graham Cormode,et al.  Node Classification in Social Networks , 2011, Social Network Data Analytics.

[8]  Feiping Nie,et al.  Cauchy Graph Embedding , 2011, ICML.

[9]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[10]  Galina Paskaleva,et al.  Non-Linear Dimensionality Reduction With Isomaps , 2012 .

[11]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[12]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[13]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[14]  Yizhou Sun,et al.  Personalized entity recommendation: a heterogeneous information network approach , 2014, WSDM.

[15]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[16]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[17]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[18]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[19]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[20]  Navdeep Jaitly,et al.  Adversarial Autoencoders , 2015, ArXiv.

[21]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[22]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[23]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[24]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[25]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[26]  Bo Zhang,et al.  Discriminative Deep Random Walk for Network Classification , 2016, ACL.