Integrating Local Vertex/Edge Embedding via Deep Matrix Fusion and Siamese Multi-label Classification

Network embedding techniques aim to encode each vertex/edge as a low-dimensional vector, enabling easy integration with existing graph mining algorithms. This paper presents a novel network embedding framework, VEEMBEDCLASS, that combines local vertex/edge embedding with deep matrix fusion and Siamese multi-label classification for facilitating classification-based local network embedding. First, we propose to perform the embeddings of each vertex/edge on K local vertex/edge embedding models respectively, with the joint optimization by considering both intra-class and inter-class correlations, to learn their latent local features on each class. The deep matrix fusion technique is developed to preserve the first-order and second-order proximity of vertices and edges on each of K classes simultaneously. Second, a Student t-distribution based Siamese multi-label classification method is designed to train associated vertices and edges with similar local characteristics together and learn their class membership probabilities, in response to the power-law vertex degree distribution widespread in real graphs. A principle of vertex-edge homophily is introduced to guarantee that the common edge/vertex shared by two associated vertices/edges and themselves are similar in terms of both structural correlations and class memberships. Finally, we integrate local vertex/edge embedding and Siamese multi-label classification into a unified model by mutually enhancing each other.

[1]  Mingchu Li,et al.  Reliable and Resilient Trust Management in Distributed Service Provision Networks , 2015, ACM Trans. Web.

[2]  Qi Zhang,et al.  GraphTwist: Fast Iterative Graph Computation with Two-tier Optimizations , 2015, Proc. VLDB Endow..

[3]  N. Latha,et al.  Personalized Recommendation Combining User Interest and Social Circle , 2015 .

[4]  Feng Qiu,et al.  Automatic identification of user interest for personalized search , 2006, WWW '06.

[5]  Hong Cheng,et al.  Graph Clustering Based on Structural/Attribute Similarities , 2009, Proc. VLDB Endow..

[6]  Hui Xiong,et al.  Understanding of Internal Clustering Validation Measures , 2010, 2010 IEEE International Conference on Data Mining.

[7]  Yizhou Sun,et al.  Personalized entity recommendation: a heterogeneous information network approach , 2014, WSDM.

[8]  Ling Liu,et al.  Social Influence Based Clustering and Optimization over Heterogeneous Information Networks , 2015, ACM Trans. Knowl. Discov. Data.

[9]  Leonid Zhukov,et al.  Joint Node-Edge Network Embedding for Link Prediction , 2018, AIST.

[10]  Yuan Zhang,et al.  COSINE: Community-Preserving Social Network Embedding From Information Diffusion Cascades , 2018, AAAI.

[11]  Yang Zhou,et al.  Dual Adversarial Learning Based Network Alignment , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[12]  Ruoming Jin,et al.  Density-aware Local Siamese Autoencoder Network Embedding with Autoencoder Graph Clustering , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[13]  Jian Pei,et al.  SNOC: Streaming Network Node Classification , 2014, 2014 IEEE International Conference on Data Mining.

[14]  Jian Pei,et al.  AutoNE: Hyperparameter Optimization for Massive Network Embedding , 2019, KDD.

[15]  Philip S. Yu,et al.  Meta path-based collective classification in heterogeneous information networks , 2012, CIKM.

[16]  Philip S. Yu,et al.  ICANE: interaction content-aware network embedding via co-embedding of nodes and edges , 2018, International Journal of Data Science and Analytics.

[17]  Shanshan Li,et al.  Deep Collective Classification in Heterogeneous Information Networks , 2018, WWW.

[18]  Bo Yang,et al.  Learning node and edge embeddings for signed networks , 2018, Neurocomputing.

[19]  M. Narasimha Murty,et al.  Structural Neighborhood Based Classification of Nodes in a Network , 2016, KDD.

[20]  Foster Provost,et al.  A Simple Relational Classifier , 2003 .

[21]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[22]  Michael K. Ng,et al.  Tensor Based Relations Ranking for Multi-relational Collective Classification , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[23]  Korris Fu-Lai Chung,et al.  A probabilistic rating inference framework for mining user preferences from reviews , 2011, World Wide Web.

[24]  Kevin Chen-Chuan Chang,et al.  Learning Community Embedding with Community Detection and Node Embedding on Graphs , 2017, CIKM.

[25]  Bin Jiang,et al.  Mining preferences from superior and inferior examples , 2008, KDD.

[26]  Yizhou Sun,et al.  Ranking-based clustering of heterogeneous information networks with star network schema , 2009, KDD.

[27]  Calton Pu,et al.  Fast Iterative Graph Computation with Resource Aware Graph Parallel Abstractions , 2015, HPDC.

[28]  Di Jin,et al.  Incorporating Network Embedding into Markov Random Field for Better Community Detection , 2019, AAAI.

[29]  Amr El Abbadi,et al.  Anonymizing Edge-Weighted Social Network Graphs , 2009 .

[30]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[31]  Gita Reese Sukthankar,et al.  Multi-label relational neighbor classification using social context features , 2013, KDD.

[32]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[33]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[34]  Jon M. Kleinberg,et al.  The link-prediction problem for social networks , 2007, J. Assoc. Inf. Sci. Technol..

[35]  Yang Zhou,et al.  Enhancing Collaborative Filtering with Multi-label Classification , 2019, CSoNet.

[36]  Jie Wang,et al.  Privacy Preservation in Social Networks with Sensitive Edge Weights , 2009, SDM.

[37]  Jiawei Han,et al.  Ranking-based classification of heterogeneous information networks , 2011, KDD.

[38]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[39]  Tijana Milenkovic,et al.  Graphlet-based edge clustering reveals pathogen-interacting proteins , 2012, Bioinform..

[40]  Xuehua Shen,et al.  Context-sensitive information retrieval using implicit feedback , 2005, SIGIR '05.

[41]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[42]  Lawrence B. Holder,et al.  Scalable SVM-Based Classification in Dynamic Graphs , 2014, 2014 IEEE International Conference on Data Mining.

[43]  L. Getoor,et al.  Link-Based Classification , 2003, Encyclopedia of Machine Learning and Data Mining.

[44]  Ling Liu,et al.  Activity-edge centric multi-label classification for mining heterogeneous information networks , 2014, KDD.

[45]  Jian Li,et al.  NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization , 2019, WWW.

[46]  Sami Abu-El-Haija,et al.  Learning Edge Representations via Low-Rank Asymmetric Projections , 2017, CIKM.

[47]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[48]  Charu C. Aggarwal,et al.  Selective sampling on graphs for classification , 2013, KDD.

[49]  David Liben-Nowell,et al.  The link-prediction problem for social networks , 2007 .

[50]  Jeffrey Xu Yu,et al.  DeepDirect: Learning Directions of Social Ties with Edge-Based Network Embedding (Extended Abstract) , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[51]  Calton Pu,et al.  Scaling iterative graph computations with GraphMap , 2015, SC15: International Conference for High Performance Computing, Networking, Storage and Analysis.

[52]  Ruoming Jin,et al.  Density-Adaptive Local Edge Representation Learning with Generative Adversarial Network Multi-label Edge Classification , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[53]  Charu C. Aggarwal,et al.  On Node Classification in Dynamic Content-based Networks , 2011, SDM.

[54]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[55]  Wei Chen,et al.  Galaxy Network Embedding: A Hierarchical Community Structure Preserving Approach , 2018, IJCAI.

[56]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[57]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[58]  Changping Wang,et al.  DeepDirect: Learning Directions of Social Ties with Edge-Based Network Embedding , 2019, IEEE Transactions on Knowledge and Data Engineering.

[59]  Peng Yang,et al.  An Aggressive Graph-Based Selective Sampling Algorithm for Classification , 2015, 2015 IEEE International Conference on Data Mining.

[60]  Ling Liu,et al.  Integrating Vertex-centric Clustering with Edge-centric Clustering for Meta Path Graph Analysis , 2015, KDD.

[61]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[62]  Charu C. Aggarwal,et al.  Link Prediction with Spatial and Temporal Consistency in Dynamic Networks , 2017, IJCAI.

[63]  Ling Liu,et al.  Social influence based clustering of heterogeneous information networks , 2013, KDD.

[64]  Philip S. Yu,et al.  Deep Recursive Network Embedding with Regular Equivalence , 2018, KDD.

[65]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[66]  Hong Cheng,et al.  Clustering Large Attributed Graphs: An Efficient Incremental Approach , 2010, 2010 IEEE International Conference on Data Mining.

[67]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[68]  Jian Pei,et al.  Arbitrary-Order Proximity Preserved Network Embedding , 2018, KDD.

[69]  Huan Liu,et al.  Scalable learning of collective behavior based on sparse social dimensions , 2009, CIKM.

[70]  Xiangliang Zhang,et al.  A Novel Framework for Node/Edge Attributed Graph Embedding , 2019, PAKDD.

[71]  Charu C. Aggarwal,et al.  Relation Strength-Aware Clustering of Heterogeneous Information Networks with Incomplete Attributes , 2012, Proc. VLDB Endow..

[72]  Hong Zhou,et al.  Geometry-Based Edge Clustering for Graph Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.