A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications

Graph is an important data representation which appears in a wide diversity of real-world scenarios. Effective graph analytics provides users a deeper understanding of what is behind the data, and thus can benefit a lot of useful applications such as node classification, node recommendation, link prediction, etc. However, most graph analytics methods suffer the high computation and space cost. Graph embedding is an effective yet efficient way to solve the graph analytics problem. It converts the graph data into a low dimensional space in which the graph structural information and graph properties are maximumly preserved. In this survey, we conduct a comprehensive review of the literature in graph embedding. We first introduce the formal definition of graph embedding as well as the related concepts. After that, we propose two taxonomies of graph embedding which correspond to what challenges exist in different graph embedding problem settings and how the existing work addresses these challenges in their solutions. Finally, we summarize the applications that graph embedding enables and suggest four promising future research directions in terms of computation efficiency, problem settings, techniques, and application scenarios.

[1]  Xuelong Li,et al.  Unsupervised Large Graph Embedding , 2017, AAAI.

[2]  Ichiro Sakata,et al.  Representation learning for geospatial areas using large-scale mobility data from smart card , 2016, UbiComp Adjunct.

[3]  Subu Surendran.  Graph Embedding and Dimensionality Reduction - A Survey , 2013 .

[4]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[5]  Zhiyuan Liu,et al.  Representation Learning of Knowledge Graphs with Entity Descriptions , 2016, AAAI.

[6]  Shaowen Wang,et al.  Regions, Periods, Activities: Uncovering Urban Dynamics via Cross-Modal Representation Learning , 2017, WWW.

[7]  Jia Wang,et al.  Topological Recurrent Neural Network for Diffusion Prediction , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[8]  Li Guo,et al.  Context-Dependent Knowledge Graph Embedding , 2015, EMNLP.

[9]  Omer Levy,et al.  Improving Distributional Similarity with Lessons Learned from Word Embeddings , 2015, TACL.

[10]  Li Guo,et al.  SSE: Semantically Smooth Embedding for Knowledge Graphs , 2017, IEEE Transactions on Knowledge and Data Engineering.

[11]  Hwann-Tzong Chen,et al.  Semantic manifold learning for image retrieval , 2005, MULTIMEDIA '05.

[12]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[13]  Feiping Nie,et al.  Nonlinear Dimensionality Reduction with Local Spline Embedding , 2009, IEEE Transactions on Knowledge and Data Engineering.

[14]  Edwin R. Hancock,et al.  Spherical and Hyperbolic Embeddings of Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Arif Mahmood,et al.  Using Geodesic Space Density Gradients for Network Community Detection , 2017, IEEE Transactions on Knowledge and Data Engineering.

[16]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[17]  T. D. Morley,et al.  Eigenvalues of the Laplacian of a graph , 1985 .

[18]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[19]  Chengqi Zhang,et al.  User Profile Preserving Social Network Embedding , 2017, IJCAI.

[20]  Tat-Seng Chua,et al.  Learning Image and User Features for Recommendation in Social Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[22]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[23]  Abdolreza Mirzaei,et al.  Hierarchical graph embedding in vector space by graph pyramid , 2017, Pattern Recognit..

[24]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[25]  Hady Wirawan Lauw,et al.  Probabilistic Latent Document Network Embedding , 2014, 2014 IEEE International Conference on Data Mining.

[26]  Jieping Ye,et al.  Hypergraph spectral learning for multi-label classification , 2008, KDD.

[27]  Zhiyuan Liu,et al.  Fast Network Embedding Enhancement via High Order Proximity Approximation , 2017, IJCAI.

[28]  Mukund Balasubramanian,et al.  The Isomap Algorithm and Topological Stability , 2002, Science.

[29]  H. Howie Huang,et al.  G-Store: High-Performance Graph Store for Trillion-Edge Processing , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[30]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[31]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[32]  Carlos Guestrin,et al.  Distributed GraphLab : A Framework for Machine Learning and Data Mining in the Cloud , 2012 .

[33]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[34]  Kilian Q. Weinberger,et al.  Learning a kernel matrix for nonlinear dimensionality reduction , 2004, ICML.

[35]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[37]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[38]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[39]  Heng Ji,et al.  Label Noise Reduction in Entity Typing by Heterogeneous Partial-Label Embedding , 2016, KDD.

[40]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[41]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[42]  Chengqi Zhang,et al.  Homophily, Structure, and Content Augmented Network Representation Learning , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[43]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[44]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[45]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[46]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[47]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[48]  Yueting Zhuang,et al.  Expert Finding for Community-Based Question Answering via Ranking Metric Network Learning , 2016, IJCAI.

[49]  Zhiyuan Liu,et al.  Knowledge Representation Learning with Entities, Attributes and Relations , 2016, IJCAI.

[50]  Alexander J. Smola,et al.  Distributed large-scale natural graph factorization , 2013, WWW.

[51]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[52]  Hanqing Lu,et al.  Community-Based Question Answering via Contextual Ranking Metric Network Learning , 2017, AAAI.

[53]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[54]  Tim Weninger,et al.  ProjE: Embedding Projection for Knowledge Graph Completion , 2016, AAAI.

[55]  Tat-Seng Chua,et al.  Learning from Collective Intelligence , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[56]  Zhiyuan Liu,et al.  Representation Learning of Knowledge Graphs with Hierarchical Types , 2016, IJCAI.

[57]  Minlie Huang,et al.  GAKE: Graph Aware Knowledge Embedding , 2016, COLING.

[58]  Ben Y. Zhao,et al.  On the Embeddability of Random Walk Distances , 2013, Proc. VLDB Endow..

[59]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[60]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[61]  James Demmel,et al.  Fast linear algebra is stable , 2006, Numerische Mathematik.

[62]  Yi Yang,et al.  Structured Embedding via Pairwise Relations and Long-Range Interactions in Knowledge Base , 2015, AAAI.

[63]  Xiaolong Jin,et al.  Predict Anchor Links across Social Networks via an Embedding Approach , 2016, IJCAI.

[64]  Wenhua Wang,et al.  Local and Global Regressive Mapping for Manifold Learning with Out-of-Sample Extrapolation , 2010, AAAI.

[65]  Cheng Li,et al.  DeepCas: An End-to-end Predictor of Information Cascades , 2016, WWW.

[66]  Carlo Zaniolo,et al.  Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment , 2016, IJCAI.

[67]  Gerhard Weikum,et al.  WWW 2007 / Track: Semantic Web Session: Ontologies ABSTRACT YAGO: A Core of Semantic Knowledge , 2022 .

[68]  Alharbi Basma,et al.  Learning from Your Network of Friends: A Trajectory Representation Learning Model Based on Online Social Ties , 2016 .

[69]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .

[70]  Jason Weston,et al.  Question Answering with Subgraph Embeddings , 2014, EMNLP.

[71]  Qing Zhang,et al.  Not All Links Are Created Equal: An Adaptive Embedding Approach for Social Personalized Ranking , 2016, SIGIR.

[72]  Pradeep Dubey,et al.  Navigating the maze of graph analytics frameworks using massive graph datasets , 2014, SIGMOD Conference.

[73]  Le Song,et al.  Discriminative Embeddings of Latent Variable Models for Structured Data , 2016, ICML.

[74]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[75]  Yufei Han,et al.  Partially Supervised Graph Embedding for Positive Unlabelled Feature Selection , 2016, IJCAI.

[76]  Yueting Zhuang,et al.  Community-Based Question Answering via Heterogeneous Social Network Learning , 2016, AAAI.

[77]  Han Xiao,et al.  From One Point to a Manifold: Knowledge Graph Embedding for Precise Link Prediction , 2015, IJCAI.

[78]  Kevin Chen-Chuan Chang,et al.  Distance-Aware DAG Embedding for Proximity Search on Heterogeneous Graphs , 2018, AAAI.

[79]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[80]  Chang Zhou,et al.  Scalable Graph Embedding for Asymmetric Proximity , 2017, AAAI.

[81]  Philip S. Yu,et al.  Cross View Link Prediction by Learning Noise-resilient Representation Consensus , 2017, WWW.

[82]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[83]  Bo Zhang,et al.  Discriminative Deep Random Walk for Network Classification , 2016, ACL.

[84]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[85]  Jason Weston,et al.  A semantic matching energy function for learning with multi-relational data , 2013, Machine Learning.

[86]  Yueting Zhuang,et al.  Community-Based Question Answering via Asymmetric Multi-Faceted Ranking Network Learning , 2017, AAAI.

[87]  Tony Jebara,et al.  Minimum Volume Embedding , 2007, AISTATS.

[88]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[89]  Kevin Chen-Chuan Chang,et al.  Subgraph-augmented Path Embedding for Semantic User Search on Heterogeneous Social Network , 2018, WWW.

[90]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[91]  Feiping Nie,et al.  Flexible Orthogonal Neighborhood Preserving Embedding , 2017, IJCAI.

[92]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[93]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[94]  James P. Callan,et al.  Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding , 2017, WWW.

[95]  Minlie Huang,et al.  SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions , 2016, AAAI.

[96]  Ivor W. Tsang,et al.  A Unified Feature Selection Framework for Graph Embedding on High Dimensional Data , 2015, IEEE Transactions on Knowledge and Data Engineering.

[97]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[98]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[99]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[100]  Tony Jebara,et al.  Structure preserving embedding , 2009, ICML '09.

[101]  Zhiyuan Liu,et al.  Max-Margin DeepWalk: Discriminative Learning of Network Representation , 2016, IJCAI.

[102]  Kevin Chen-Chuan Chang,et al.  From Node Embedding To Community Embedding , 2016, ArXiv.

[103]  Yuanzhuo Wang,et al.  Locally Adaptive Translation for Knowledge Graph Embedding , 2015, AAAI.

[104]  Jun Zhao,et al.  Knowledge Graph Embedding via Dynamic Mapping Matrix , 2015, ACL.

[105]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[106]  Meng Wang,et al.  Learning content–social influential features for influence analysis , 2016, International Journal of Multimedia Information Retrieval.

[107]  Kevin Chen-Chuan Chang,et al.  Learning Community Embedding with Community Detection and Node Embedding on Graphs , 2017, CIKM.

[108]  Li Guo,et al.  Semantically Smooth Knowledge Graph Embedding , 2015, ACL.

[109]  Jiawei Han,et al.  Large-Scale Embedding Learning in Heterogeneous Event Data , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[110]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[111]  Jie Tang,et al.  Multi-Modal Bayesian Embeddings for Learning Social Knowledge Graphs , 2015, IJCAI.

[112]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[113]  Dacheng Tao,et al.  Signed Laplacian Embedding for Supervised Dimension Reduction , 2014, AAAI.

[114]  Rui Zhang,et al.  Incorporating Knowledge Graph Embeddings into Topic Modeling , 2017, AAAI.

[115]  Yueting Zhuang,et al.  Learning of Multimodal Representations With Random Walks on the Click Graph , 2016, IEEE Transactions on Image Processing.

[116]  Adriano Veloso,et al.  Unsupervised and Scalable Algorithm for Learning Node Representations , 2017, ICLR.

[117]  Devdatt P. Dubhashi,et al.  Learning with Similarity Functions on Graphs using Matchings of Geometric Embeddings , 2015, KDD.

[118]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[119]  Li Wen,et al.  Dimensionality reduction on Anchorgraph with an efficient Locality Preserving Projection , 2016, Neurocomputing.

[120]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[121]  Ichiro Sakata,et al.  Geospatial Area Embedding Based on the Movement Purpose Hypothesis Using Large-Scale Mobility Data from Smart Card , 2016 .

[122]  Mohamed Nadif,et al.  A Semi-NMF-PCA Unified Framework for Data Clustering , 2017, IEEE Transactions on Knowledge and Data Engineering.

[123]  Jianxin Li,et al.  On the Representation and Embedding of Knowledge Bases beyond Binary Relations , 2016, IJCAI.

[124]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[125]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[126]  Jun Zhao,et al.  Knowledge Graph Completion with Adaptive Sparse Transfer Matrix , 2016, AAAI.

[127]  Yizhou Yu,et al.  Piecewise Flat Embedding for Image Segmentation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[128]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[129]  John A Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD , 2012, Behavior research methods.

[130]  Enhong Chen,et al.  Learning Deep Representations for Graph Clustering , 2014, AAAI.

[131]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[132]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[133]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[134]  Pascal Frossard,et al.  Graph-based Isometry Invariant Representation Learning , 2017, ICML.

[135]  Junbin Gao,et al.  Laplacian Regularized Low-Rank Representation and Its Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[136]  Michalis Vazirgiannis,et al.  Matching Node Embeddings for Graph Similarity , 2017, AAAI.

[137]  Joachim M. Buhmann,et al.  Multidimensional Scaling and Data Clustering , 1994, NIPS.

[138]  S. V. N. Vishwanathan,et al.  Graph kernels , 2007 .

[139]  Juan-Zi Li,et al.  Text-Enhanced Representation Learning for Knowledge Graph , 2016, IJCAI.

[140]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[141]  Li Liu,et al.  Aligning Users across Social Networks Using Network Embedding , 2016, IJCAI.

[142]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[143]  Feiping Nie,et al.  Capped Lp-Norm Graph Embedding for Photo Clustering , 2016, ACM Multimedia.

[144]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[145]  Kevin Chen-Chuan Chang,et al.  Semantic Proximity Search on Heterogeneous Graph by Proximity Embedding , 2017, AAAI.

[146]  Zhiyuan Liu,et al.  Representation Learning for Measuring Entity Relatedness with Rich Information , 2015, IJCAI.

[147]  Jens Lehmann,et al.  DBpedia - A crystallization point for the Web of Data , 2009, J. Web Semant..

[148]  Jiawei Han,et al.  Spectral regression: a unified subspace learning framework for content-based image retrieval , 2007, ACM Multimedia.

[149]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[150]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[151]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[152]  Daniel Dajun Zeng,et al.  Predicting user's multi-interests with network embedding in health-related topics , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[153]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[154]  Han Xiao,et al.  TransG : A Generative Model for Knowledge Graph Embedding , 2015, ACL.

[155]  Reynold Xin,et al.  GraphX: Graph Processing in a Distributed Dataflow Framework , 2014, OSDI.