Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks

Graph Neural Networks (GNNs) have been widely used for the representation learning of various structured graph data, typically through message passing among nodes by aggregating their neighborhood information via different operations. While promising, most existing GNNs oversimplify the complexity and diversity of the edges in the graph and thus are inefficient to cope with ubiquitous heterogeneous graphs, which are typically in the form of multi-relational graph representations. In this article, we propose RioGNN, a novel Reinforced, recursive, and flexible neighborhood selection guided multi-relational Graph Neural Network architecture, to navigate complexity of neural network structures whilst maintaining relation-dependent representations. We first construct a multi-relational graph, according to the practical task, to reflect the heterogeneity of nodes, edges, attributes, and labels. To avoid the embedding over-assimilation among different types of nodes, we employ a label-aware neural similarity measure to ascertain the most similar neighbors based on node attributes. A reinforced relation-aware neighbor selection mechanism is developed to choose the most similar neighbors of a targeting node within a relation before aggregating all neighborhood information from different relations to obtain the eventual node embedding. Particularly, to improve the efficiency of neighbor selecting, we propose a new recursive and scalable reinforcement learning framework with estimable depth and width for different scales of multi-relational graphs. RioGNN can learn more discriminative node embedding with enhanced explainability due to the recognition of individual importance of each relation via the filtering threshold mechanism. Comprehensive experiments on real-world graph data and practical tasks demonstrate the advancements of effectiveness, efficiency, and the model explainability, as opposed to other comparative GNN models.

[1]  Philip S. Yu,et al.  A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources , 2020, IEEE Transactions on Big Data.

[2]  Philip S. Yu,et al.  Lifelong Property Price Prediction: A Case Study for the Toronto Real Estate Market , 2020, IEEE Transactions on Knowledge and Data Engineering.

[3]  Philip S. Yu,et al.  Lime: Low-Cost and Incremental Learning for Dynamic Heterogeneous Information Networks , 2021, IEEE Transactions on Computers.

[4]  A. Cabellos-Aparicio,et al.  Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case , 2019, Comput. Commun..

[5]  Yu Zheng,et al.  Predicting Citywide Crowd Flows in Irregular Regions Using Multi-View Graph Convolutional Networks , 2019, IEEE Transactions on Knowledge and Data Engineering.

[6]  Philip S. Yu,et al.  Adversarial Attack and Defense on Graph Data: A Survey , 2018 .

[7]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[8]  Jie Xu,et al.  Hawk: Rapid Android Malware Detection Through Heterogeneous Graph Attention Networks , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Philip S. Yu,et al.  Streaming Social Event Detection and Evolution Discovery in Heterogeneous Information Networks , 2021, ACM Trans. Knowl. Discov. Data.

[10]  Hao Peng,et al.  KGSynNet: A Novel Entity Synonyms Discovery Framework with Knowledge Graph , 2021, DASFAA.

[11]  Philip S. Yu,et al.  Knowledge-Preserving Incremental Social Event Detection via Heterogeneous GNNs , 2021, WWW.

[12]  Hao Peng,et al.  SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism , 2021, WWW.

[13]  Zehua Guo,et al.  Combining Deep Reinforcement Learning With Graph Neural Networks for Optimal VNF Placement , 2021, IEEE Communications Letters.

[14]  Hong Chen,et al.  Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation , 2020, WSDM.

[15]  Philip S. Yu,et al.  KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning , 2020, AAAI.

[16]  Yang Wang,et al.  Survey on Deep Multi-modal Data Analytics: Collaboration, Rivalry, and Fusion , 2020, ACM Trans. Multim. Comput. Commun. Appl..

[17]  Philip S. Yu,et al.  Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification , 2019, IEEE Transactions on Knowledge and Data Engineering.

[18]  Hao Peng,et al.  Federated Knowledge Graphs Embedding , 2021, ArXiv.

[19]  Philip S. Yu,et al.  Heterogeneous Similarity Graph Neural Network on Electronic Health Records , 2020, 2020 IEEE International Conference on Big Data (Big Data).

[20]  Cheng-te Li,et al.  Reinforcement Learning Enhanced Heterogeneous Graph Neural Network , 2020, ArXiv.

[21]  T. Pevný,et al.  Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks , 2020, ArXiv.

[22]  Yang Wang,et al.  Block-Aware Item Similarity Models for Top-N Recommendation , 2020, ACM Trans. Inf. Syst..

[23]  Philip S. Yu,et al.  Pairwise Learning for Name Disambiguation in Large-Scale Heterogeneous Academic Networks , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[24]  Maoguo Gong,et al.  MGAT: Multi-view Graph Attention Networks , 2020, Neural Networks.

[25]  Philip S. Yu,et al.  Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters , 2020, CIKM.

[26]  Xia Hu,et al.  Policy-GNN: Aggregation Optimization for Graph Neural Networks , 2020, KDD.

[27]  Philip S. Yu,et al.  Attentional Graph Convolutional Networks for Knowledge Concept Recommendation in MOOCs in a Heterogeneous View , 2020, SIGIR.

[28]  Alois Knoll,et al.  Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments , 2020, 2020 IEEE Intelligent Vehicles Symposium (IV).

[29]  Philip S. Yu,et al.  Robust Spammer Detection by Nash Reinforcement Learning , 2020, KDD.

[30]  Zi Huang,et al.  Exploiting Cross-session Information for Session-based Recommendation with Graph Neural Networks , 2020, ACM Trans. Inf. Syst..

[31]  Zi Huang,et al.  GCN-Based User Representation Learning for Unifying Robust Recommendation and Fraudster Detection , 2020, SIGIR.

[32]  Selim F. Yilmaz,et al.  Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization , 2020, ArXiv.

[33]  Philip S. Yu,et al.  Alleviating the Inconsistency Problem of Applying Graph Neural Network to Fraud Detection , 2020, SIGIR.

[34]  Philip S. Yu,et al.  Multi-information Source HIN for Medical Concept Embedding , 2020, PAKDD.

[35]  Senzhang Wang,et al.  Motif-Matching Based Subgraph-Level Attentional Convolutional Network for Graph Classification , 2020, AAAI.

[36]  Ville Hautamäki,et al.  Action Space Shaping in Deep Reinforcement Learning , 2020, 2020 IEEE Conference on Games (CoG).

[37]  Yizhou Sun,et al.  Heterogeneous Graph Transformer , 2020, WWW.

[38]  Pushmeet Kohli,et al.  Unveiling the predictive power of static structure in glassy systems , 2020 .

[39]  Irwin King,et al.  MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding , 2020, WWW.

[40]  Philip S. Yu,et al.  Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting , 2020, Inf. Sci..

[41]  Jieping Ye,et al.  An Attention-based Graph Neural Network for Heterogeneous Structural Learning , 2019, AAAI.

[42]  Yu Chen,et al.  Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings , 2019, NeurIPS.

[43]  Mohammed J. Zaki,et al.  Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation , 2019, ICLR.

[44]  Senzhang Wang,et al.  Label-Aware Graph Convolutional Networks , 2019, CIKM.

[45]  Michael W. Dusenberry,et al.  Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer , 2019, AAAI.

[46]  Mohammed J. Zaki,et al.  Deep Iterative and Adaptive Learning for Graph Neural Networks , 2019, ArXiv.

[47]  Yanfang Ye,et al.  Key Player Identification in Underground Forums over Attributed Heterogeneous Information Network Embedding Framework , 2019, CIKM.

[48]  Jun Zhou,et al.  A Semi-Supervised Graph Attentive Network for Financial Fraud Detection , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[49]  Yingcheng Sun,et al.  Opinion Spam Detection Based on Heterogeneous Information Network , 2019, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI).

[50]  Chuan Zhou,et al.  Relation Structure-Aware Heterogeneous Graph Neural Network , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[51]  Le Song,et al.  Neural Similarity Learning , 2019, NeurIPS.

[52]  Yoshua Bengio,et al.  GraphMix: Regularized Training of Graph Neural Networks for Semi-Supervised Learning , 2019, ArXiv.

[53]  Yu He,et al.  HeteSpaceyWalk: A Heterogeneous Spacey Random Walk for Heterogeneous Information Network Embedding , 2019, CIKM.

[54]  Dong Li,et al.  Spam Review Detection with Graph Convolutional Networks , 2019, CIKM.

[55]  Nitesh V. Chawla,et al.  Heterogeneous Graph Neural Network , 2019, KDD.

[56]  M. de Rijke,et al.  Bayesian Personalized Feature Interaction Selection for Factorization Machines , 2019, SIGIR.

[57]  Weixiong Zhang,et al.  Graph Convolutional Networks Meet Markov Random Fields: Semi-Supervised Community Detection in Attribute Networks , 2019, AAAI.

[58]  Ning Feng,et al.  Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting , 2019, AAAI.

[59]  Yuzhong Qu,et al.  Multi-view Knowledge Graph Embedding for Entity Alignment , 2019, IJCAI.

[60]  Manohar Kaul,et al.  Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs , 2019, ACL.

[61]  Philip S. Yu,et al.  Fine-grained Event Categorization with Heterogeneous Graph Convolutional Networks , 2019, IJCAI.

[62]  Kai Yu,et al.  AgentGraph: Toward Universal Dialogue Management With Structured Deep Reinforcement Learning , 2019, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[63]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.

[64]  Yu Huang,et al.  FdGars: Fraudster Detection via Graph Convolutional Networks in Online App Review System , 2019, WWW.

[65]  Yang Gao,et al.  GraphNAS: Graph Neural Architecture Search with Reinforcement Learning , 2019, ArXiv.

[66]  Yanfang Ye,et al.  Heterogeneous Graph Attention Network , 2019, WWW.

[67]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[68]  Gong Zhang,et al.  GCN-GAN: A Non-linear Temporal Link Prediction Model for Weighted Dynamic Networks , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[69]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[70]  Xiaolong Li,et al.  GeniePath: Graph Neural Networks with Adaptive Receptive Paths , 2018, AAAI.

[71]  Yingtong Dou A Review of Recent Advance in Online Spam Detection , 2019 .

[72]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[73]  Henry Zhu,et al.  Soft Actor-Critic Algorithms and Applications , 2018, ArXiv.

[74]  Jiawei Han,et al.  Meta-Graph Based HIN Spectral Embedding: Methods, Analyses, and Insights , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[75]  Le Song,et al.  Heterogeneous Graph Neural Networks for Malicious Account Detection , 2018, CIKM.

[76]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[77]  Fei Wang,et al.  Multi-View Graph Convolutional Network and Its Applications on Neuroimage Analysis for Parkinson's Disease , 2018, AMIA.

[78]  Li Zhao,et al.  Reinforcement Learning for Relation Classification From Noisy Data , 2018, AAAI.

[79]  Yanchun Zhang,et al.  Community Detection in Attributed Graphs: An Embedding Approach , 2018, AAAI.

[80]  Majid Sarrafzadeh,et al.  HeteroMed: Heterogeneous Information Network for Medical Diagnosis , 2018, CIKM.

[81]  Jianxin Li,et al.  Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN , 2018, WWW.

[82]  Fei Wang,et al.  Drug Similarity Integration Through Attentive Multi-view Graph Auto-Encoders , 2018, IJCAI.

[83]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[84]  Herke van Hoof,et al.  Addressing Function Approximation Error in Actor-Critic Methods , 2018, ICML.

[85]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[86]  Seyed Mehran Kazemi,et al.  SimplE Embedding for Link Prediction in Knowledge Graphs , 2018, NeurIPS.

[87]  Anna Cinzia Squicciarini,et al.  Combating Crowdsourced Review Manipulators: A Neighborhood-Based Approach , 2018, WSDM.

[88]  Ruoyu Li,et al.  Adaptive Graph Convolutional Neural Networks , 2018, AAAI.

[89]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[90]  Xiaohui Liang,et al.  Smoke Screener or Straight Shooter: Detecting Elite Sybil Attacks in User-Review Social Networks , 2017, NDSS.

[91]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[92]  Philip S. Yu,et al.  Multi-view Clustering with Graph Embedding for Connectome Analysis , 2017, CIKM.

[93]  Philip S. Yu,et al.  HitFraud: A Broad Learning Approach for Collective Fraud Detection in Heterogeneous Information Networks , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[94]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[95]  Wenhan Xiong,et al.  DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning , 2017, EMNLP.

[96]  Jure Leskovec,et al.  Predicting multicellular function through multi-layer tissue networks , 2017, Bioinform..

[97]  Kaiming He,et al.  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour , 2017, ArXiv.

[98]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[99]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[100]  Mohammad Al Hasan,et al.  Name Disambiguation in Anonymized Graphs using Network Embedding , 2017, CIKM.

[101]  Maruf Pasha,et al.  Survey of Machine Learning Algorithms for Disease Diagnostic , 2017 .

[102]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[103]  Philip S. Yu,et al.  A Survey of Heterogeneous Information Network Analysis , 2015, IEEE Transactions on Knowledge and Data Engineering.

[104]  Christina Eldredge,et al.  Population Analysis of Adverse Events in Different Age Groups Using Big Clinical Trials Data , 2016, JMIR medical informatics.

[105]  Anazida Zainal,et al.  Fraud detection system: A survey , 2016, J. Netw. Comput. Appl..

[106]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[107]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[108]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[109]  Richard Evans,et al.  Deep Reinforcement Learning in Large Discrete Action Spaces , 2015, 1512.07679.

[110]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[111]  Leman Akoglu,et al.  Collective Opinion Spam Detection: Bridging Review Networks and Metadata , 2015, KDD.

[112]  Reynold Cheng,et al.  Discovering Meta-Paths in Large Heterogeneous Information Networks , 2015, WWW.

[113]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[114]  Arjun Mukherjee,et al.  What Yelp Fake Review Filter Might Be Doing? , 2013, ICWSM.

[115]  Jure Leskovec,et al.  From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews , 2013, WWW.

[116]  Graham Cormode,et al.  Node Classification in Social Networks , 2011, Social Network Data Analytics.

[117]  J. Contreras,et al.  Finding Multiple Nash Equilibria in Pool-Based Markets: A Stochastic EPEC Approach , 2011, IEEE Transactions on Power Systems.

[118]  T. Murata,et al.  Advanced modularity-specialized label propagation algorithm for detecting communities in networks , 2009, 0910.1154.

[119]  Steve Gregory,et al.  Finding overlapping communities in networks by label propagation , 2009, ArXiv.

[120]  Réka Albert,et al.  Near linear time algorithm to detect community structures in large-scale networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[121]  George Karypis,et al.  Comparison of descriptor spaces for chemical compound retrieval and classification , 2006, Sixth International Conference on Data Mining (ICDM'06).

[122]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[123]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[124]  Mark Newman,et al.  Detecting community structure in networks , 2004 .

[125]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[126]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[127]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[128]  Vijay R. Konda,et al.  Actor-Critic Algorithms , 1999, NIPS.

[129]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.