Multi-Task Learning Based Network Embedding

The goal of network representation learning, also called network embedding, is to encode the network structure information into a continuous low-dimensionality embedding space where geometric relationships among the vectors can reflect the relationships of nodes in the original network. The existing network representation learning methods are always single-task learning, in which case these methods focus on preserving the proximity of nodes from one aspect. However, the proximity of nodes is dependent on both the local and global structure, resulting in a limitation on the node embeddings learned by these methods. In order to solve this problem, in this paper, we propose a novel method, Multi-Task Learning-Based Network Embedding, termed MLNE. There are two tasks in this method so as to preserve the proximity of nodes. The aim of the first task is to preserve the high-order proximity between pairwise nodes in the whole network. The second task is to preserve the low-order proximity in the one-hop area of each node. By jointly learning these tasks in the supervised deep learning model, our method can obtain node embeddings that can sufficiently reflect the roles that nodes play in networks. In order to demonstrate the efficacy of our MLNE method over existing state-of-the-art methods, we conduct experiments on multi-label classification, link prediction, and visualization in five real-world networks. The experimental results show that our method performs competitively.

[1]  Tsuyoshi Murata,et al.  A General View for Network Embedding as Matrix Factorization , 2019, WSDM.

[2]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[3]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[4]  Steven Skiena,et al.  A Tutorial on Network Embeddings , 2018, ArXiv.

[5]  Yue Yin,et al.  Explainable Recommendation via Multi-Task Learning in Opinionated Text Data , 2018, SIGIR.

[6]  Philip S. Yu,et al.  Heterogeneous Information Network Embedding for Recommendation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[7]  Charu C. Aggarwal,et al.  An embedding approach to anomaly detection , 2016, 2016 IEEE 32nd International Conference on Data Engineering (ICDE).

[8]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[9]  Katarzyna Musial,et al.  Properties of Bridge Nodes in Social Networks , 2009, ICCCI.

[10]  Bo Du,et al.  Multi-Task Learning for Blind Source Separation , 2018, IEEE Transactions on Image Processing.

[11]  Maoguo Gong,et al.  Computational Intelligence for Network Structure Analytics , 2017, Springer Singapore.

[12]  Junwei Han,et al.  SPFTN: A Joint Learning Framework for Localizing and Segmenting Objects in Weakly Labeled Videos , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[14]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[15]  Kim-Han Thung,et al.  A brief review on multi-task learning , 2018, Multimedia Tools and Applications.

[16]  Luis Argerich,et al.  Hash2Vec, Feature Hashing for Word Embeddings , 2016, ArXiv.

[17]  Linyuan Lu,et al.  Link Prediction in Complex Networks: A Survey , 2010, ArXiv.

[18]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[19]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[20]  Chengqi Zhang,et al.  Network Representation Learning: A Survey , 2017, IEEE Transactions on Big Data.

[21]  Jian Li,et al.  NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization , 2019, WWW.

[22]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[23]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[24]  Qiang Yang,et al.  An Overview of Multi-task Learning , 2018 .

[25]  Jie Tang,et al.  ArnetMiner: extraction and mining of academic social networks , 2008, KDD.

[26]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[27]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.

[28]  Sophie Ahrens,et al.  Recommender Systems , 2012 .

[29]  Ivan Herman,et al.  Graph Visualization and Navigation in Information Visualization: A Survey , 2000, IEEE Trans. Vis. Comput. Graph..

[30]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[31]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[32]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[33]  Ken Lang,et al.  NewsWeeder: Learning to Filter Netnews , 1995, ICML.

[34]  Philip S. Yu,et al.  Cross View Link Prediction by Learning Noise-resilient Representation Consensus , 2017, WWW.

[35]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[36]  Andrew McCallum,et al.  Automating the Construction of Internet Portals with Machine Learning , 2000, Information Retrieval.

[37]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[38]  Xiaodong Liu,et al.  Multi-Task Deep Neural Networks for Natural Language Understanding , 2019, ACL.

[39]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.