A Tutorial on Network Embeddings

Network embedding methods aim at learning low-dimensional latent representation of nodes in a network. These representations can be used as features for a wide range of tasks on graphs such as classification, clustering, link prediction, and visualization. In this survey, we give an overview of network embeddings by summarizing and categorizing recent advancements in this research field. We first discuss the desirable properties of network embeddings and briefly introduce the history of network embedding algorithms. Then, we discuss network embedding methods under different scenarios, such as supervised versus unsupervised learning, learning embeddings for homogeneous networks versus for heterogeneous networks, etc. We further demonstrate the applications of network embeddings, and conclude the survey with future work in this area.

[1]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[2]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[3]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[4]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[5]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[6]  Yifan Hu,et al.  Efficient, High-Quality Force-Directed Graph Drawing , 2006 .

[7]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[8]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.

[9]  Huan Liu,et al.  Leveraging social media networks for classification , 2011, Data Mining and Knowledge Discovery.

[10]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[11]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[12]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[13]  Omer Levy,et al.  Neural Word Embedding as Implicit Matrix Factorization , 2014, NIPS.

[14]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[15]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[16]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[17]  Tat-Seng Chua,et al.  Learning Image and User Features for Recommendation in Social Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Daniel Jurafsky,et al.  Learning multi-faceted representations of individuals from heterogeneous evidence using neural networks , 2015, ArXiv.

[19]  Zhiyuan Liu,et al.  Representation Learning for Measuring Entity Relatedness with Rich Information , 2015, IJCAI.

[20]  Yang Yang,et al.  Learning Features from Large-Scale, Noisy and Social Image-Tag Collection , 2015, ACM Multimedia.

[21]  Yi-Hsuan Yang,et al.  Do You Have a Pop Face? Here is a Pop Song. Using Profile Pictures to Mitigate the Cold-start Problem in Music Recommender Systems , 2015, RecSys Posters.

[22]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[23]  Steven Skiena,et al.  Exact Age Prediction in Social Networks , 2015, WWW.

[24]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[25]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[26]  Yueting Zhuang,et al.  Community-Based Question Answering via Heterogeneous Social Network Learning , 2016, AAAI.

[27]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[28]  Jiang Guo,et al.  A General Framework for Content-enhanced Network Representation Learning , 2016, ArXiv.

[29]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[30]  Andreas Hotho,et al.  Extracting Semantics from Random Walks on Wikipedia: Comparing Learning and Counting Methods , 2016, Wiki@ICWSM.

[31]  Heng Ji,et al.  Label Noise Reduction in Entity Typing by Heterogeneous Partial-Label Embedding , 2016, KDD.

[32]  Xuanjing Huang,et al.  Incorporate Group Information to Enhance Network Embedding , 2016, CIKM.

[33]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[34]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[35]  Chengqi Zhang,et al.  Homophily, Structure, and Content Augmented Network Representation Learning , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[36]  Tat-Seng Chua,et al.  Learning from Collective Intelligence , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[37]  Zhiyuan Liu,et al.  Max-Margin DeepWalk: Discriminative Learning of Network Representation , 2016, IJCAI.

[38]  Jie Tang,et al.  Multi-Modal Bayesian Embeddings for Learning Social Knowledge Graphs , 2015, IJCAI.

[39]  Yi-Hsuan Yang,et al.  Query-based Music Recommendations via Preference Embedding , 2016, RecSys.

[40]  Yueting Zhuang,et al.  Learning of Multimodal Representations With Random Walks on the Click Graph , 2016, IEEE Transactions on Image Processing.

[41]  Heiko Paulheim,et al.  RDF2Vec: RDF Graph Embeddings for Data Mining , 2016, SEMWEB.

[42]  Yueting Zhuang,et al.  Expert Finding for Community-Based Question Answering via Ranking Metric Network Learning , 2016, IJCAI.

[43]  Jiawei Han,et al.  Large-Scale Embedding Learning in Heterogeneous Event Data , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[44]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[45]  Li Liu,et al.  Aligning Users across Social Networks Using Network Embedding , 2016, IJCAI.

[46]  Alex Alemi,et al.  Watch Your Step: Learning Graph Embeddings Through Attention , 2017, ArXiv.

[47]  Zhiyuan Liu,et al.  CANE: Context-Aware Network Embedding for Relation Modeling , 2017, ACL.

[48]  Steven Skiena,et al.  Vector-based similarity measurements for historical figures , 2017, Inf. Syst..

[49]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[50]  Chang Zhou,et al.  Scalable Graph Embedding for Asymmetric Proximity , 2017, AAAI.

[51]  Christos Faloutsos,et al.  Fast, Warped Graph Embedding: Unifying Framework and One-Click Algorithm , 2017, ArXiv.

[52]  Steven Skiena,et al.  DeepBrowse: Similarity-Based Browsing Through Large Lists (Extended Abstract) , 2017, SISAP.

[53]  Philip S. Yu,et al.  Embedding of Embedding (EOE): Joint Embedding for Coupled Heterogeneous Networks , 2017, WSDM.

[54]  Sami Abu-El-Haija,et al.  Learning Edge Representations via Low-Rank Asymmetric Projections , 2017, CIKM.

[55]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[56]  Charu C. Aggarwal,et al.  Signed Network Embedding in Social Media , 2017, SDM.

[57]  Zhiyuan Liu,et al.  TransNet: Translation-Based Network Representation Learning for Social Relation Extraction , 2017, IJCAI.

[58]  Zhiyuan Liu,et al.  A Neural Network Approach to Joint Modeling Social Networks and Mobile Trajectories , 2016, ArXiv.

[59]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[60]  Steven Skiena,et al.  Don't Walk, Skip!: Online Learning of Multi-scale Network Embeddings , 2016, ASONAM.

[61]  Yang Xiang,et al.  SNE: Signed Network Embedding , 2017, PAKDD.

[62]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[63]  Steven Skiena,et al.  HARP: Hierarchical Representation Learning for Networks , 2017, AAAI.