Network representation learning: models, methods and applications

With the rise of large-scale social networks, network mining has become an important sub-domain of data mining. Generating an efficient network representation is one important challenge in applying machine learning to network data. Recently, representation learning methods are widely used in various domains to generate low dimensional latent features from complex high dimensional data. A significant amount of research effort is made in the past few years to generate node representations from graph-structured data using representation learning methods. Here, we provide a detailed study of the latest advancements in the field of network representation learning (also called network embedding). We first discuss the basic concepts and models of network embedding. Further, we build a taxonomy of network embedding methods based on the type of networks and review the major research works that come under each category. We then cover the major datasets used in network embedding research and describe the major applications of network embedding with respect to various network mining tasks. Finally, we provide various directions for future work which enhance further research.

[1]  Jari Saramäki,et al.  Temporal Networks , 2011, Encyclopedia of Social Network Analysis and Mining.

[2]  Zhiyuan Liu,et al.  Max-Margin DeepWalk: Discriminative Learning of Network Representation , 2016, IJCAI.

[3]  Nagarajan Natarajan,et al.  Inductive matrix completion for predicting gene–disease associations , 2014, Bioinform..

[4]  Jon M. Kleinberg,et al.  Overview of the 2003 KDD Cup , 2003, SKDD.

[5]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[6]  Yang Xiang,et al.  SNE: Signed Network Embedding , 2017, PAKDD.

[7]  Wanjiun Liao,et al.  Tracking Network Evolution and Their Applications in Structural Network Analysis , 2019, IEEE Transactions on Network Science and Engineering.

[8]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[9]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[10]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[11]  Bo Zhang,et al.  Discriminative Deep Random Walk for Network Classification , 2016, ACL.

[12]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[13]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[14]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[15]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[16]  Jie Tang,et al.  ArnetMiner: extraction and mining of academic social networks , 2008, KDD.

[17]  Jennifer Neville,et al.  Attributed graph models: modeling network structure with correlated attributes , 2014, WWW.

[18]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[19]  Cecilia Mascolo,et al.  Graph Metrics for Temporal Networks , 2013, ArXiv.

[20]  Tina Eliassi-Rad,et al.  Leveraging Label-Independent Features for Classification in Sparsely Labeled Networks: An Empirical Study , 2008, SNAKDD.

[21]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Graham Cormode,et al.  Node Classification in Social Networks , 2011, Social Network Data Analytics.

[23]  Yizhou Sun,et al.  Task-Guided and Path-Augmented Heterogeneous Network Embedding for Author Identification , 2016, WSDM.

[24]  Aram Galstyan,et al.  Scalable Temporal Latent Space Inference for Link Prediction in Dynamic Social Networks (Extended Abstract) , 2017, 2017 IEEE 33rd International Conference on Data Engineering (ICDE).

[25]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[26]  S. Shen-Orr,et al.  Network motifs: simple building blocks of complex networks. , 2002, Science.

[27]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[28]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[29]  Satu Elisa Schaeffer,et al.  Graph Clustering , 2017, Encyclopedia of Machine Learning and Data Mining.

[30]  Kara Dolinski,et al.  The BioGRID Interaction Database: 2008 update , 2008, Nucleic Acids Res..

[31]  Junghwan Kim,et al.  SIDE: Representation Learning in Signed Directed Networks , 2018, WWW.

[32]  Mathieu Bastian,et al.  Gephi: An Open Source Software for Exploring and Manipulating Networks , 2009, ICWSM.

[33]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[34]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[35]  Dong Yu,et al.  Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[36]  Linyuan Lu,et al.  Link Prediction in Complex Networks: A Survey , 2010, ArXiv.

[37]  Mohammad Al Hasan,et al.  Name Disambiguation in Anonymized Graphs using Network Embedding , 2017, CIKM.

[38]  Charu C. Aggarwal,et al.  Attributed Signed Network Embedding , 2017, CIKM.

[39]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[40]  Roger M. Needham The changing environment for security protocols , 1997 .

[41]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[42]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[43]  Philip S. Yu,et al.  Multi-Label Collective Classification , 2011, SDM.

[44]  Tobias Müller,et al.  Identifying functional modules in protein–protein interaction networks: an integrated exact approach , 2008, ISMB.

[45]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[46]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[47]  Sophie Ahrens,et al.  Recommender Systems , 2012 .

[48]  S. V. N. Vishwanathan,et al.  Graph kernels , 2007 .

[49]  Minyi Guo,et al.  SHINE: Signed Heterogeneous Information Network Embedding for Sentiment Link Prediction , 2017, WSDM.

[50]  Omer Levy,et al.  Improving Distributional Similarity with Lessons Learned from Word Embeddings , 2015, TACL.

[51]  Xiaolong Jin,et al.  Predict Anchor Links across Social Networks via an Embedding Approach , 2016, IJCAI.

[52]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[53]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[54]  Hanna M. Wallach,et al.  Computational social science and social computing , 2013, Machine Learning.

[55]  C. Steglich,et al.  DYNAMIC NETWORKS AND BEHAVIOR: SEPARATING SELECTION FROM INFLUENCE: separating selection from influence , 2010 .

[56]  Jovica V. Milanović,et al.  Interdepedency modeling of cyber-physical systems using a weighted complex network approach , 2017, 2017 IEEE Manchester PowerTech.

[57]  Xiao Huang,et al.  Exploring Expert Cognition for Attributed Network Embedding , 2018, WSDM.

[58]  Zhiyuan Liu,et al.  TransNet: Translation-Based Network Representation Learning for Social Relation Extraction , 2017, IJCAI.

[59]  Claudio Martella,et al.  Practical Graph Analytics with Apache Giraph , 2015, Apress.

[60]  Zhiyuan Liu,et al.  CANE: Context-Aware Network Embedding for Relation Modeling , 2017, ACL.

[61]  Yoshua Bengio,et al.  Deep Learning for NLP (without Magic) , 2012, ACL.

[62]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[63]  Philip S. Yu,et al.  On Exploring Semantic Meanings of Links for Embedding Social Networks , 2018, WWW.

[64]  Luis G. Moyano,et al.  Learning network representations , 2017, The European Physical Journal Special Topics.

[65]  Yizhou Sun,et al.  Mining Heterogeneous Information Networks: Principles and Methodologies , 2012, Mining Heterogeneous Information Networks: Principles and Methodologies.

[66]  Steven Skiena,et al.  Vector-based similarity measurements for historical figures , 2017, Inf. Syst..

[67]  Xiao Huang,et al.  Label Informed Attributed Network Embedding , 2017, WSDM.

[68]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[69]  Tao Wu,et al.  Predicting the evolution of complex networks via local information , 2016, ArXiv.

[70]  Andrew McCallum,et al.  Automating the Construction of Internet Portals with Machine Learning , 2000, Information Retrieval.

[71]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[72]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[73]  Jennifer Neville,et al.  Iterative Classification in Relational Data , 2000 .

[74]  Tony Jebara,et al.  Structure preserving embedding , 2009, ICML '09.

[75]  Akira Utsumi,et al.  A Complex Network Approach to Distributional Semantic Models , 2015, PloS one.

[76]  Jiawei Han,et al.  Curriculum Learning for Heterogeneous Star Network Embedding via Deep Reinforcement Learning , 2018, WSDM.

[77]  Xiao Huang,et al.  Accelerated Attributed Network Embedding , 2017, SDM.

[78]  Giorgio Valle,et al.  Scuba: scalable kernel-based gene prioritization , 2018, BMC Bioinformatics.

[79]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[80]  Yueting Zhuang,et al.  Dynamic Network Embedding by Modeling Triadic Closure Process , 2018, AAAI.

[81]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.

[82]  Yiming Yang,et al.  The Enron Corpus: A New Dataset for Email Classi(cid:12)cation Research , 2004 .

[83]  Alessio Micheli,et al.  Neural Network for Graphs: A Contextual Constructive Approach , 2009, IEEE Transactions on Neural Networks.

[84]  Charu C. Aggarwal,et al.  Signed Network Embedding in Social Media , 2017, SDM.

[85]  Chun-Ta Lu,et al.  HEER: Heterogeneous graph embedding for emerging relation detection from news , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[86]  Jure Leskovec,et al.  {SNAP Datasets}: {Stanford} Large Network Dataset Collection , 2014 .

[87]  Huan Liu,et al.  Scalable learning of collective behavior based on sparse social dimensions , 2009, CIKM.

[88]  Alexander J. Smola,et al.  Distributed large-scale natural graph factorization , 2013, WWW.

[89]  Daniel Dajun Zeng,et al.  Predicting user's multi-interests with network embedding in health-related topics , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[90]  Alessandro Sperduti,et al.  On Filter Size in Graph Convolutional Networks , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[91]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[92]  Avi Ma’ayan Introduction to Network Analysis in Systems Biology , 2011, Science Signaling.

[93]  Huan Liu,et al.  Attributed Network Embedding for Learning in a Dynamic Environment , 2017, CIKM.

[94]  David Liben-Nowell,et al.  The link-prediction problem for social networks , 2007 .

[95]  Reynold Xin,et al.  GraphX: a resilient distributed graph system on Spark , 2013, GRADES.

[96]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[97]  Ludovic Denoyer,et al.  Learning social network embeddings for predicting information diffusion , 2014, WSDM.

[98]  Wang-Chien Lee,et al.  HIN2Vec: Explore Meta-paths in Heterogeneous Information Networks for Representation Learning , 2017, CIKM.

[99]  Jure Leskovec,et al.  Signed networks in social media , 2010, CHI.

[100]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[101]  Charu C. Aggarwal,et al.  A Survey of Signed Network Mining in Social Media , 2015, ACM Comput. Surv..

[102]  Michalis Vazirgiannis,et al.  Locating influential nodes in complex networks , 2016, Scientific Reports.

[103]  A. Barabasi,et al.  Network medicine : a network-based approach to human disease , 2010 .

[104]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[105]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[106]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[107]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[108]  V. S. Subrahmanian,et al.  VEWS: A Wikipedia Vandal Early Warning System , 2015, KDD.

[109]  Jiawei Han,et al.  Large-Scale Embedding Learning in Heterogeneous Event Data , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[110]  Philip S. Yu,et al.  PathSim , 2011, Proc. VLDB Endow..

[111]  J. Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: A computational study , 2007, Behavior research methods.

[112]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[113]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[114]  Ryan A. Rossi,et al.  Continuous-Time Dynamic Network Embeddings , 2018, WWW.

[115]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[116]  Alessandro Sperduti,et al.  Multiple Graph-Kernel Learning , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[117]  Chirag Shah,et al.  Collaborative User Network Embedding for Social Recommender Systems , 2017, SDM.

[118]  Heng Ji,et al.  Exploring Context and Content Links in Social Media: A Latent Space Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[119]  Yun Fu,et al.  Graph Embedding for Pattern Analysis , 2012 .