Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks

Graph Neural Networks (GNNs) have been widely used for the representation learning of various structured graph data, typically through message passing among nodes by aggregating their neighborhood information via different operations. While promising, most existing GNNs oversimplify the complexity and diversity of the edges in the graph and thus are inefficient to cope with ubiquitous heterogeneous graphs, which are typically in the form of multi-relational graph representations. In this article, we propose RioGNN, a novel Reinforced, recursive, and flexible neighborhood selection guided multi-relational Graph Neural Network architecture, to navigate complexity of neural network structures whilst maintaining relation-dependent representations. We first construct a multi-relational graph, according to the practical task, to reflect the heterogeneity of nodes, edges, attributes, and labels. To avoid the embedding over-assimilation among different types of nodes, we employ a label-aware neural similarity measure to ascertain the most similar neighbors based on node attributes. A reinforced relation-aware neighbor selection mechanism is developed to choose the most similar neighbors of a targeting node within a relation before aggregating all neighborhood information from different relations to obtain the eventual node embedding. Particularly, to improve the efficiency of neighbor selecting, we propose a new recursive and scalable reinforcement learning framework with estimable depth and width for different scales of multi-relational graphs. RioGNN can learn more discriminative node embedding with enhanced explainability due to the recognition of individual importance of each relation via the filtering threshold mechanism. Comprehensive experiments on real-world graph data and practical tasks demonstrate the advancements of effectiveness, efficiency, and the model explainability, as opposed to other comparative GNN models.

[1]  Yanfang Ye,et al.  Heterogeneous Graph Attention Network , 2019, WWW.

[2]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[3]  Jun Zhou,et al.  A Semi-Supervised Graph Attentive Network for Financial Fraud Detection , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[4]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[5]  Philip S. Yu,et al.  Multi-information Source HIN for Medical Concept Embedding , 2020, PAKDD.

[6]  Xia Hu,et al.  Policy-GNN: Aggregation Optimization for Graph Neural Networks , 2020, KDD.

[7]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.

[8]  Zi Huang,et al.  Exploiting Cross-session Information for Session-based Recommendation with Graph Neural Networks , 2020, ACM Trans. Inf. Syst..

[9]  Senzhang Wang,et al.  Motif-Matching Based Subgraph-Level Attentional Convolutional Network for Graph Classification , 2020, AAAI.

[10]  George Karypis,et al.  Comparison of descriptor spaces for chemical compound retrieval and classification , 2006, Sixth International Conference on Data Mining (ICDM'06).

[11]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[12]  Yizhou Sun,et al.  Heterogeneous Graph Transformer , 2020, WWW.

[13]  T. Pevný,et al.  Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks , 2020, ArXiv.

[14]  Kaiming He,et al.  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour , 2017, ArXiv.

[15]  Yuzhong Qu,et al.  Multi-view Knowledge Graph Embedding for Entity Alignment , 2019, IJCAI.

[16]  Xiaolong Li,et al.  GeniePath: Graph Neural Networks with Adaptive Receptive Paths , 2018, AAAI.

[17]  Henry Zhu,et al.  Soft Actor-Critic Algorithms and Applications , 2018, ArXiv.

[18]  Yang Gao,et al.  GraphNAS: Graph Neural Architecture Search with Reinforcement Learning , 2019, ArXiv.

[19]  Philip S. Yu,et al.  Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters , 2020, CIKM.

[20]  Alois Knoll,et al.  Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments , 2020, 2020 IEEE Intelligent Vehicles Symposium (IV).

[21]  Yoshua Bengio,et al.  GraphMix: Regularized Training of Graph Neural Networks for Semi-Supervised Learning , 2019, ArXiv.

[22]  Fei Wang,et al.  Drug Similarity Integration Through Attentive Multi-view Graph Auto-Encoders , 2018, IJCAI.

[23]  Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case. , 2019, 1910.07421.

[24]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[25]  Yu Chen,et al.  Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings , 2019, NeurIPS.

[26]  Philip S. Yu,et al.  Pairwise Learning for Name Disambiguation in Large-Scale Heterogeneous Academic Networks , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[27]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[28]  Philip S. Yu,et al.  Adversarial Attack and Defense on Graph Data: A Survey , 2018 .

[29]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[30]  Ning Feng,et al.  Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting , 2019, AAAI.

[31]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[32]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[33]  Richard Evans,et al.  Deep Reinforcement Learning in Large Discrete Action Spaces , 2015, 1512.07679.

[34]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[35]  Yu Huang,et al.  FdGars: Fraudster Detection via Graph Convolutional Networks in Online App Review System , 2019, WWW.

[36]  Ville Hautamäki,et al.  Action Space Shaping in Deep Reinforcement Learning , 2020, 2020 IEEE Conference on Games (CoG).

[37]  Reynold Cheng,et al.  Discovering Meta-Paths in Large Heterogeneous Information Networks , 2015, WWW.

[38]  Maoguo Gong,et al.  MGAT: Multi-view Graph Attention Networks , 2020, Neural Networks.

[39]  Zi Huang,et al.  GCN-Based User Representation Learning for Unifying Robust Recommendation and Fraudster Detection , 2020, SIGIR.

[40]  Zehua Guo,et al.  Combining Deep Reinforcement Learning With Graph Neural Networks for Optimal VNF Placement , 2021, IEEE Communications Letters.

[41]  Yingcheng Sun,et al.  Opinion Spam Detection Based on Heterogeneous Information Network , 2019, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI).

[42]  Philip S. Yu,et al.  Lifelong Property Price Prediction: A Case Study for the Toronto Real Estate Market , 2020, IEEE Transactions on Knowledge and Data Engineering.

[43]  Yingtong Dou A Review of Recent Advance in Online Spam Detection , 2019 .

[44]  Maruf Pasha,et al.  Survey of Machine Learning Algorithms for Disease Diagnostic , 2017 .

[45]  Jiawei Han,et al.  Meta-Graph Based HIN Spectral Embedding: Methods, Analyses, and Insights , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[46]  M. de Rijke,et al.  Bayesian Personalized Feature Interaction Selection for Factorization Machines , 2019, SIGIR.

[47]  Jure Leskovec,et al.  From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews , 2013, WWW.

[48]  Herke van Hoof,et al.  Addressing Function Approximation Error in Actor-Critic Methods , 2018, ICML.

[49]  Jie Xu,et al.  Hawk: Rapid Android Malware Detection Through Heterogeneous Graph Attention Networks , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Anazida Zainal,et al.  Fraud detection system: A survey , 2016, J. Netw. Comput. Appl..

[51]  Ruoyu Li,et al.  Adaptive Graph Convolutional Neural Networks , 2018, AAAI.

[52]  Michael W. Dusenberry,et al.  Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer , 2019, AAAI.

[53]  Yu Zheng,et al.  Predicting Citywide Crowd Flows in Irregular Regions Using Multi-View Graph Convolutional Networks , 2019, IEEE Transactions on Knowledge and Data Engineering.

[54]  Christina Eldredge,et al.  Population Analysis of Adverse Events in Different Age Groups Using Big Clinical Trials Data , 2016, JMIR medical informatics.

[55]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[56]  Reinforcement Learning Enhanced Heterogeneous Graph Neural Network , 2020, ArXiv.

[57]  Hao Peng,et al.  KGSynNet: A Novel Entity Synonyms Discovery Framework with Knowledge Graph , 2021, DASFAA.

[58]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[59]  Majid Sarrafzadeh,et al.  HeteroMed: Heterogeneous Information Network for Medical Diagnosis , 2018, CIKM.

[60]  Kai Yu,et al.  AgentGraph: Toward Universal Dialogue Management With Structured Deep Reinforcement Learning , 2019, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[61]  Dong Li,et al.  Spam Review Detection with Graph Convolutional Networks , 2019, CIKM.

[62]  Réka Albert,et al.  Near linear time algorithm to detect community structures in large-scale networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[63]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[64]  J. Contreras,et al.  Finding Multiple Nash Equilibria in Pool-Based Markets: A Stochastic EPEC Approach , 2011, IEEE Transactions on Power Systems.

[65]  Arjun Mukherjee,et al.  What Yelp Fake Review Filter Might Be Doing? , 2013, ICWSM.

[66]  Graham Cormode,et al.  Node Classification in Social Networks , 2011, Social Network Data Analytics.

[67]  Jieping Ye,et al.  An Attention-based Graph Neural Network for Heterogeneous Structural Learning , 2019, AAAI.

[68]  Seyed Mehran Kazemi,et al.  SimplE Embedding for Link Prediction in Knowledge Graphs , 2018, NeurIPS.

[69]  Le Song,et al.  Neural Similarity Learning , 2019, NeurIPS.

[70]  Philip S. Yu,et al.  Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting , 2020, Inf. Sci..

[71]  Xiaohui Liang,et al.  Smoke Screener or Straight Shooter: Detecting Elite Sybil Attacks in User-Review Social Networks , 2017, NDSS.

[72]  Philip S. Yu,et al.  Lime: Low-Cost and Incremental Learning for Dynamic Heterogeneous Information Networks , 2021, IEEE Transactions on Computers.

[73]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[74]  Philip S. Yu,et al.  HitFraud: A Broad Learning Approach for Collective Fraud Detection in Heterogeneous Information Networks , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[75]  Hong Chen,et al.  Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation , 2020, WSDM.

[76]  Anna Cinzia Squicciarini,et al.  Combating Crowdsourced Review Manipulators: A Neighborhood-Based Approach , 2018, WSDM.

[77]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[78]  Yanfang Ye,et al.  Key Player Identification in Underground Forums over Attributed Heterogeneous Information Network Embedding Framework , 2019, CIKM.

[79]  Senzhang Wang,et al.  Label-Aware Graph Convolutional Networks , 2019, CIKM.

[80]  Wenhan Xiong,et al.  DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning , 2017, EMNLP.

[81]  Philip S. Yu,et al.  Robust Spammer Detection by Nash Reinforcement Learning , 2020, KDD.

[82]  Mark Newman,et al.  Detecting community structure in networks , 2004 .

[83]  Philip S. Yu,et al.  Alleviating the Inconsistency Problem of Applying Graph Neural Network to Fraud Detection , 2020, SIGIR.

[84]  Fei Wang,et al.  Multi-View Graph Convolutional Network and Its Applications on Neuroimage Analysis for Parkinson's Disease , 2018, AMIA.

[85]  Hao Peng,et al.  Federated Knowledge Graphs Embedding , 2021, ArXiv.

[86]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[87]  Philip S. Yu,et al.  Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification , 2019, IEEE Transactions on Knowledge and Data Engineering.

[88]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[89]  Philip S. Yu,et al.  KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning , 2020, AAAI.

[90]  Leman Akoglu,et al.  Collective Opinion Spam Detection: Bridging Review Networks and Metadata , 2015, KDD.

[91]  Philip S. Yu,et al.  A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources , 2020, IEEE Transactions on Big Data.

[92]  Mohammad Al Hasan,et al.  Name Disambiguation in Anonymized Graphs using Network Embedding , 2017, CIKM.

[93]  Jianxin Li,et al.  Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN , 2018, WWW.

[94]  Philip S. Yu,et al.  Fine-grained Event Categorization with Heterogeneous Graph Convolutional Networks , 2019, IJCAI.

[95]  Le Song,et al.  Heterogeneous Graph Neural Networks for Malicious Account Detection , 2018, CIKM.

[96]  Philip S. Yu,et al.  A Survey of Heterogeneous Information Network Analysis , 2015, IEEE Transactions on Knowledge and Data Engineering.

[97]  Vijay R. Konda,et al.  Actor-Critic Algorithms , 1999, NIPS.

[98]  Yanchun Zhang,et al.  Community Detection in Attributed Graphs: An Embedding Approach , 2018, AAAI.

[99]  Li Zhao,et al.  Reinforcement Learning for Relation Classification From Noisy Data , 2018, AAAI.

[100]  Jure Leskovec,et al.  Predicting multicellular function through multi-layer tissue networks , 2017, Bioinform..

[101]  Pushmeet Kohli,et al.  Unveiling the predictive power of static structure in glassy systems , 2020 .

[102]  Yu He,et al.  HeteSpaceyWalk: A Heterogeneous Spacey Random Walk for Heterogeneous Information Network Embedding , 2019, CIKM.

[103]  Yang Wang,et al.  Block-Aware Item Similarity Models for Top-N Recommendation , 2020, ACM Trans. Inf. Syst..

[104]  Hao Peng,et al.  SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism , 2021, WWW.

[105]  Irwin King,et al.  MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding , 2020, WWW.

[106]  Chuan Zhou,et al.  Relation Structure-Aware Heterogeneous Graph Neural Network , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[107]  Weixiong Zhang,et al.  Graph Convolutional Networks Meet Markov Random Fields: Semi-Supervised Community Detection in Attribute Networks , 2019, AAAI.

[108]  Philip S. Yu,et al.  Attentional Graph Convolutional Networks for Knowledge Concept Recommendation in MOOCs in a Heterogeneous View , 2020, SIGIR.

[109]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[110]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[111]  Philip S. Yu,et al.  Heterogeneous Similarity Graph Neural Network on Electronic Health Records , 2020, 2020 IEEE International Conference on Big Data (Big Data).

[112]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[113]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[114]  Mohammed J. Zaki,et al.  Deep Iterative and Adaptive Learning for Graph Neural Networks , 2019, ArXiv.

[115]  Philip S. Yu,et al.  Knowledge-Preserving Incremental Social Event Detection via Heterogeneous GNNs , 2021, WWW.

[116]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[117]  Philip S. Yu,et al.  Multi-view Clustering with Graph Embedding for Connectome Analysis , 2017, CIKM.

[118]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[119]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[120]  T. Murata,et al.  Advanced modularity-specialized label propagation algorithm for detecting communities in networks , 2009, 0910.1154.

[121]  Nitesh V. Chawla,et al.  Heterogeneous Graph Neural Network , 2019, KDD.

[122]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[123]  Mohammed J. Zaki,et al.  Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation , 2019, ICLR.

[124]  Philip S. Yu,et al.  Streaming Social Event Detection and Evolution Discovery in Heterogeneous Information Networks , 2021, ACM Trans. Knowl. Discov. Data.

[125]  Yang Wang,et al.  Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry, and Fusion , 2020, ACM Trans. Multim. Comput. Commun. Appl..

[126]  Steve Gregory,et al.  Finding overlapping communities in networks by label propagation , 2009, ArXiv.

[127]  Selim F. Yilmaz,et al.  Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization , 2020, ArXiv.

[128]  Gong Zhang,et al.  GCN-GAN: A Non-linear Temporal Link Prediction Model for Weighted Dynamic Networks , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[129]  Manohar Kaul,et al.  Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs , 2019, ACL.