MLNE: Multi-Label Network Embedding

Network embedding aims to preserve topological structures of a network using low-dimensional vectors and has shown to be effective for driving a myriad of graph mining tasks (e.g., link prediction or classification) free of the stressful feature extraction procedure. Many methods have been proposed to integrate node content and/or label information, with nodes sharing similar content/labels being close to each other in the learned latent space. To date, existing methods either consider networked instances with a single label or consider a set of labels as a whole for node representation learning. Therefore, they cannot handle network of instances containing multiple labels (i.e. multi-labels), which are ubiquitous in describing complex concepts of instances. In this article, we formulate a new multi-label network embedding (MLNE) problem to learn feature representation for networked multi-label instances. We argue that the key to MLNE learning is to aggregate node topology structures, node content, and multi-label correlations. We propose a two-layer network embedding framework to couple information for effective learning. To capture higher order label correlations, we use labels to form a high-level label–label network over a low-level node–node network, in which the label network interacts with the node network through multi-labeling relations. The low-level node–node network can be enhanced by latent label-specific features from high-level label network with well-captured high-order correlations between labels. To enable the multi-label informed network embedding, we force both node and label representations being optimized under the same low-dimensional latent space by a unified training objective. Experiments on real-world data sets demonstrate that MLNE achieves better performance compared with methods with or without considering label information.

[1]  Aapo Hyvärinen,et al.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models , 2010, AISTATS.

[2]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[3]  David M. Blei,et al.  Relational Topic Models for Document Networks , 2009, AISTATS.

[4]  Michael Eickenberg,et al.  Machine learning for neuroimaging with scikit-learn , 2014, Front. Neuroinform..

[5]  Shunxiang Wu,et al.  Multi-label learning based on label-specific features and local pairwise label correlation , 2018, Neurocomputing.

[6]  D. Tao,et al.  A modified stochastic neighbor embedding for multi-feature dimension reduction of remote sensing images , 2013 .

[7]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[8]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[9]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[10]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[11]  Charu C. Aggarwal,et al.  You Are How You Drive: Peer and Temporal-Aware Representation Learning for Driving Behavior Analysis , 2018, KDD.

[12]  Marianne Winslett,et al.  Understanding Social Causalities Behind Human Action Sequences , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Huan Liu,et al.  Multi-Layered Network Embedding , 2018, SDM.

[14]  Xiaolin Li,et al.  Learning Urban Community Structures , 2018, ACM Trans. Intell. Syst. Technol..

[15]  Xuanjing Huang,et al.  Incorporate Group Information to Enhance Network Embedding , 2016, CIKM.

[16]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Philip S. Yu,et al.  Multiple Structure-View Learning for Graph Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Zhi-Hua Zhou,et al.  Multi-Label Learning with Global and Local Label Correlation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[19]  Dacheng Tao,et al.  Grassmannian Regularized Structured Multi-View Embedding for Image Classification , 2013, IEEE Transactions on Image Processing.

[20]  Xiao Huang,et al.  Label Informed Attributed Network Embedding , 2017, WSDM.

[21]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[22]  Jiang Guo,et al.  A General Framework for Content-enhanced Network Representation Learning , 2016, ArXiv.

[23]  Jianxun Liu,et al.  A Topic-Sensitive Method for Mashup Tag Recommendation Utilizing Multi-Relational Service Data , 2021, IEEE Transactions on Services Computing.

[24]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[25]  Hady Wirawan Lauw,et al.  Probabilistic Latent Document Network Embedding , 2014, 2014 IEEE International Conference on Data Mining.

[26]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[28]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[29]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[30]  Maoguo Gong,et al.  Learning to Map Social Network Users by Unified Manifold Alignment on Hypergraph , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Bo Du,et al.  Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding , 2015, Pattern Recognit..

[32]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[33]  Bo Du,et al.  Target detection based on a dynamic subspace , 2014, Pattern Recognit..

[34]  Yihong Gong,et al.  A Two-Level Topic Model Towards Knowledge Discovery from Citation Networks , 2014, IEEE Transactions on Knowledge and Data Engineering.

[35]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[36]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[37]  Chengqi Zhang,et al.  Network Representation Learning: A Survey , 2017, IEEE Transactions on Big Data.

[38]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[39]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.