Learning Bi-typed Multi-relational Heterogeneous Graph via Dual Hierarchical Attention Networks

Bi-type multi-relational heterogeneous graph (BMHG) is one of the most common graphs in practice, for example, academic networks, e-commerce user behavior graph and enterprise knowledge graph. It is a critical and challenge problem on how to learn the numerical representation for each node to characterize subtle structures. However, most previous studies treat all node relations in BMHG as the same class of relation without distinguishing the different characteristics between the intra-class relations and inter-class relations of the bi-typed nodes, causing the loss of significant structure information. To address this issue, we propose a novel Dual Hierarchical Attention Networks (DHAN) based on the bi-typed multi-relational heterogeneous graphs to learn comprehensive node representations with the intra-class and inter-class attention-based encoder under a hierarchical mechanism. Specifically, the former encoder aggregates information from the same type of nodes, while the latter aggregates node representations from its different types of neighbors. Moreover, to sufficiently model node multi-relational information in BMHG, we adopt a newly proposed hierarchical mechanism. By doing so, the proposed dual hierarchical attention operations enable our model to fully capture the complex structures of the bi-typed multi-relational heterogeneous graphs. Experimental results on various tasks against the state-of-the-arts sufficiently confirm the capability of DHAN in learning node representations on the BMHGs.

[1]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[2]  Ryan A. Rossi,et al.  Graph Classification using Structural Attention , 2018, KDD.

[3]  Philip S. Yu,et al.  Influence and similarity on heterogeneous networks , 2012, CIKM.

[4]  Ryan A. Rossi,et al.  Graph Convolutional Networks with Motif-based Attention , 2019, CIKM.

[5]  Jianxun Lian,et al.  Self-supervised Graph Learning for Recommendation , 2020, SIGIR.

[6]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[7]  Yizhou Sun,et al.  Heterogeneous Graph Transformer , 2020, WWW.

[8]  Shouwei Li,et al.  Systemic risk in bank-firm multiplex networks , 2020 .

[9]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[10]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[11]  Deng Cai,et al.  Translating Embeddings for Knowledge Graph Completion with Relation Attention Mechanism , 2018, IJCAI.

[12]  Hongbo Deng,et al.  A Dual Heterogeneous Graph Attention Network to Improve Long-Tail Performance for Shop Search in E-Commerce , 2020, KDD.

[13]  Xiaoyan Zhu,et al.  Commonsense Knowledge Aware Conversation Generation with Graph Attention , 2018, IJCAI.

[14]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[15]  Nitesh V. Chawla,et al.  Heterogeneous Graph Neural Network , 2019, KDD.

[16]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[17]  Kai Wang,et al.  Relational Graph Attention Network for Aspect-based Sentiment Analysis , 2020, ACL.

[18]  Linmei Hu,et al.  Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification , 2019, EMNLP.

[19]  Yorick Wilks,et al.  A Closer Look at Skip-gram Modelling , 2006, LREC.

[20]  Jorge Cadima,et al.  Principal component analysis: a review and recent developments , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[21]  Philip S. Yu,et al.  A Survey of Heterogeneous Information Network Analysis , 2015, IEEE Transactions on Knowledge and Data Engineering.

[22]  Jun Wang,et al.  Adaptive Structural Fingerprints for Graph Attention Networks , 2020, ICLR.

[23]  Vikram Nitin,et al.  Composition-based Multi-Relational Graph Convolutional Networks , 2020, ICLR.

[24]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[25]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[26]  Jieping Ye,et al.  PINE: Universal Deep Embedding for Graph Nodes via Partial Permutation Invariant Set Functions , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Philip S. Yu,et al.  A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources , 2020, IEEE Transactions on Big Data.

[28]  Jingrui He,et al.  DEMO-Net: Degree-specific Graph Neural Networks for Node and Graph Classification , 2019, KDD.

[29]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[30]  Jaewoo Kang,et al.  Graph Transformer Networks , 2019, NeurIPS.

[31]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[32]  C.-C. Jay Kuo,et al.  Graph representation learning: a survey , 2019, APSIPA Transactions on Signal and Information Processing.

[33]  Geoffrey J. Gordon,et al.  Relational learning via collective matrix factorization , 2008, KDD.

[34]  Hongzhi Chen,et al.  Measuring and Improving the Use of Graph Information in Graph Neural Networks , 2020, ICLR.

[35]  Rong Jin,et al.  Understanding bag-of-words model: a statistical framework , 2010, Int. J. Mach. Learn. Cybern..

[36]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[37]  Zhirong Liu,et al.  Dual Graph enhanced Embedding Neural Network for CTR Prediction , 2021, KDD.

[38]  Weifeng Lv,et al.  Hybrid Micro/Macro Level Convolution for Heterogeneous Graph Learning , 2020, ArXiv.

[39]  Alexander A. Alemi,et al.  Watch Your Step: Learning Node Embeddings via Graph Attention , 2017, NeurIPS.

[40]  Yaliang Li,et al.  Simple and Deep Graph Convolutional Networks , 2020, ICML.

[41]  Giulio Cimini,et al.  Statistically validated network of portfolio overlaps and systemic risk , 2016, Scientific Reports.

[42]  Minlie Huang,et al.  GAKE: Graph Aware Knowledge Embedding , 2016, COLING.

[43]  Valentina Y. Guleva,et al.  Using Multiplex Networks for Banking Systems Dynamics Modelling , 2015 .

[44]  Pietro Cavallo,et al.  Relational Graph Attention Networks , 2018, ArXiv.

[45]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[46]  Pengfei Liu,et al.  Heterogeneous Graph Neural Networks for Extractive Document Summarization , 2020, ACL.

[47]  Ryan A. Rossi,et al.  Attention Models in Graphs: A Survey , 2018 .

[48]  Dong Li,et al.  Spam Review Detection with Graph Convolutional Networks , 2019, CIKM.

[49]  Yanfang Ye,et al.  Heterogeneous Graph Attention Network , 2019, WWW.

[50]  Xing Xie,et al.  Relation-Aware Graph Convolutional Networks for Agent-Initiated Social E-Commerce Recommendation , 2019, CIKM.

[51]  Jianxin Li,et al.  Interpretable and Efficient Heterogeneous Graph Convolutional Network , 2021, IEEE Transactions on Knowledge and Data Engineering.

[52]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[53]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.