CAGE: Constrained deep Attributed Graph Embedding

Abstract In this paper we deal with complex attributed graphs which can exhibit rich connectivity patterns and whose nodes are often associated with attributes, such as text or images. In order to analyze these graphs, the primary challenge is to find an effective way to represent them by preserving both structural properties and node attribute information. To create low-dimensional and meaningful embedded representations of these complex graphs, we propose a fully unsupervised model based on Deep Learning architectures, called Constrained Attributed Graph Embedding model (CAGE). The main contribution of the proposed model is the definition of a novel two-phase optimization problem that explicitly models node attributes to obtain a higher representation expressiveness while preserving the local and the global structural properties of the graph. We validated our approach on two different benchmark datasets for node classification. Experimental results demonstrate that this novel representation provides significant improvements compared to state of the art approaches, also showing higher robustness with respect to the size of the training data.

[1]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[2]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[3]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[4]  Tim S. Evans,et al.  Modelling citation networks , 2014, Scientometrics.

[5]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[6]  David D. Cox,et al.  Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures , 2013, ICML.

[7]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[8]  Alberto Montresor,et al.  gat2vec: representation learning for attributed graphs , 2018, Computing.

[9]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[10]  Xiao Huang,et al.  Accelerated Attributed Network Embedding , 2017, SDM.

[11]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[12]  Patrick Gallinari,et al.  A Latent Representation Model for Sentiment Analysis in Heterogeneous Social Networks , 2014, SEFM Workshops.

[13]  Yanchun Zhang,et al.  Community Detection in Attributed Graphs: An Embedding Approach , 2018, AAAI.

[14]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[15]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[16]  Philip S. Yu,et al.  Embedding of Embedding (EOE): Joint Embedding for Coupled Heterogeneous Networks , 2017, WSDM.

[17]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[18]  Hong Cheng,et al.  Clustering Large Attributed Graphs: An Efficient Incremental Approach , 2010, 2010 IEEE International Conference on Data Mining.

[19]  Palash Goyal,et al.  GEM: A Python package for graph embedding methods , 2018, J. Open Source Softw..

[20]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[21]  Palash Goyal,et al.  Capturing Edge Attributes via Network Embedding , 2018, IEEE Transactions on Computational Social Systems.

[22]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[23]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[24]  Huan Liu,et al.  Attributed Network Embedding for Learning in a Dynamic Environment , 2017, CIKM.

[25]  Steven Skiena,et al.  HARP: Hierarchical Representation Learning for Networks , 2017, AAAI.

[26]  Wenwu Zhu,et al.  DepthLGP: Learning Embeddings of Out-of-Sample Nodes in Dynamic Networks , 2018, AAAI.

[27]  Wray L. Buntine,et al.  Bibliographic Analysis with the Citation Network Topic Model , 2016, ACML.

[28]  Jie Tang,et al.  ArnetMiner: extraction and mining of academic social networks , 2008, KDD.

[29]  Francesco Archetti,et al.  A probabilistic relational approach for web document clustering , 2010, Inf. Process. Manag..

[30]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[31]  Yu Chen,et al.  KATE: K-Competitive Autoencoder for Text , 2017, KDD.

[32]  Alexander J. Smola,et al.  Distributed large-scale natural graph factorization , 2013, WWW.

[33]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[34]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[35]  Ludovic Denoyer,et al.  Representation Learning for Classification in Heterogeneous Graphs with Application to Social Networks , 2018, ACM Trans. Knowl. Discov. Data.

[36]  Le Wu,et al.  Deep Attributed Network Embedding by Preserving Structure and Attribute Information , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[37]  Tat-Seng Chua,et al.  Learning from Collective Intelligence , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[38]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..