Co-Embedding Network Nodes and Hierarchical Labels with Taxonomy Based Generative Adversarial Networks

Network embedding aims at transferring node proximity in networks into distributed vectors, which can be leveraged in various downstream applications. Recent research has shown that nodes in a network can often be organized in latent hierarchical structures, but without a particular underlying taxonomy, the learned node embedding is less useful nor interpretable. In this work, we aim to improve network embedding by modeling the conditional node proximity in networks indicated by node labels residing in real taxonomies. In the meantime, we also aim to model the hierarchical label proximity in the given taxonomies, which is too coarse by solely looking at the hierarchical topologies. To this end, we propose TaxoGAN to co-embed network nodes and hierarchical labels, through a hierarchical network generation process. Particularly, TaxoGAN models the child labels and network nodes of each parent label in an individual embedding space while learning to transfer network proximity among the spaces of hierarchical labels through stacked network generators and embedding encoders. To enable robust and efficient model inference, we further develop a hierarchical adversarial training process. Comprehensive experiments and case studies on four real-world datasets of networks with hierarchical labels demonstrate the utility of TaxoGAN in improving network embedding on traditional tasks of node classification and link prediction, as well as novel tasks like conditional proximity search and fine-grained taxonomy layout.

[1]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[2]  Neil Genzlinger A. and Q , 2006 .

[3]  W. Marsden I and J , 2012 .

[4]  Gerhard Weikum,et al.  PATTY: A Taxonomy of Relational Patterns with Semantic Types , 2012, EMNLP.

[5]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Jiawei Han,et al.  Constructing topical hierarchies in heterogeneous information networks , 2013, 2013 IEEE 13th International Conference on Data Mining.

[7]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[8]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[9]  Haixun Wang,et al.  Learning Term Embeddings for Hypernymy Identification , 2015, IJCAI.

[10]  Yi Yang,et al.  Efficient Methods for Inferring Large Sparse Topic Hierarchies , 2015, ACL.

[11]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[12]  Erik Cambria,et al.  Label Embedding for Zero-shot Fine-grained Named Entity Typing , 2016, COLING.

[13]  Zhuo Chen,et al.  Deep clustering: Discriminative embeddings for segmentation and separation , 2015, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[15]  Cordelia Schmid,et al.  Label-Embedding for Image Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Siu Cheung Hui,et al.  Learning Term Embeddings for Taxonomic Relation Identification Using Dynamic Weighting Neural Network , 2016, EMNLP.

[17]  Xiao Huang,et al.  Accelerated Attributed Network Embedding , 2017, SDM.

[18]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[19]  Ngoc Thang Vu,et al.  Hierarchical Embeddings for Hypernymy Detection and Directionality , 2017, EMNLP.

[20]  Lantao Yu,et al.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient , 2016, AAAI.

[21]  Donna K. Slonim,et al.  Towards a more molecular taxonomy of disease , 2016, J. Biomed. Semant..

[22]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[23]  Xiao Huang,et al.  Label Informed Attributed Network Embedding , 2017, WSDM.

[24]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[25]  Jiawei Han,et al.  An Attention-based Collaboration Framework for Multi-View Network Representation Learning , 2017, CIKM.

[26]  John E. Hopcroft,et al.  Stacked Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[28]  Jiawei Han,et al.  MetaPAD: Meta Pattern Discovery from Massive Text Corpora , 2017, KDD.

[29]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[30]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[31]  Jiajun Bu,et al.  ANRL: Attributed Network Representation Learning via Deep Neural Networks , 2018, IJCAI.

[32]  Guoyin Wang,et al.  Joint Embedding of Words and Labels for Text Classification , 2018, ACL.

[33]  Rodrigo C. Barros,et al.  Hierarchical Multi-Label Classification Networks , 2018, ICML.

[34]  Jianxin Li,et al.  Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN , 2018, WWW.

[35]  Ivan Vulić,et al.  Specialising Word Vectors for Lexical Entailment , 2017, NAACL.

[36]  Brian M. Sadler,et al.  TaxoGen: Unsupervised Topic Taxonomy Construction by Adaptive Term Embedding and Clustering , 2018, KDD.

[37]  Xiao Wang,et al.  Hierarchical Taxonomy Aware Network Embedding , 2018, KDD.

[38]  Charu C. Aggarwal,et al.  Learning Deep Network Representations with Adversarially Regularized Autoencoders , 2018, KDD.

[39]  Minyi Guo,et al.  GraphGAN: Graph Representation Learning with Generative Adversarial Nets , 2017, AAAI.

[40]  Xiao Huang,et al.  On Interpretation of Network Embedding via Taxonomy Induction , 2018, KDD.

[41]  Danushka Bollegala,et al.  Joint Learning of Hierarchical Word Embeddings from a Corpus and a Taxonomy , 2019, AKBC.

[42]  Yizhou Sun,et al.  Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts , 2019, KDD.

[43]  Jian Pei,et al.  ProGAN: Network Embedding via Proximity Generative Adversarial Network , 2019, KDD.

[44]  Xinbing Wang,et al.  CommunityGAN: Community Detection with Generative Adversarial Nets , 2019, WWW.

[45]  Qiang Li,et al.  Adversarial Training Methods for Network Embedding , 2019, WWW.

[46]  Xiangliang Zhang,et al.  Co-Embedding Attributed Networks , 2019, WSDM.

[47]  Zhenan Sun,et al.  A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications , 2020, IEEE Transactions on Knowledge and Data Engineering.