ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network

Graph Neural Networks (GNNs) have been widely studied in various graph data mining tasks. Most existing GNNs embed graph data into Euclidean space and thus are less effective to capture the ubiquitous hierarchical structures in real-world networks. Hyperbolic Graph Neural Networks (HGNNs) extend GNNs to hyperbolic space and thus are more effective to capture the hierarchical structures of graphs in node representation learning. In hyperbolic geometry, the graph hierarchical structure can be reflected by the curvatures of the hyperbolic space, and different curvatures can model different hierarchical structures of a graph. However, most existing HGNNs manually set the curvature to a fixed value for simplicity, which achieves a suboptimal performance of graph learning due to the complex and diverse hierarchical structures of the graphs. To resolve this problem, we propose an Adaptive Curvature Exploration Hyperbolic Graph Neural Network named ACE-HGNN to adaptively learn the optimal curvature according to the input graph and downstream tasks. Specifically, ACE-HGNN exploits a multi-agent reinforcement learning framework and contains two agents, ACE-Agent and HGNN-Agent for learning the curvature and node representations, respectively. The two agents are updated by a Nash Q-leaning algorithm collaboratively, seeking the optimal hyperbolic space indexed by the curvature. Extensive experiments on multiple real-world graph datasets demonstrate a significant and consistent performance improvement in model quality with competitive performance and good generalization ability.

[1]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[2]  Philip S. Yu,et al.  Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters , 2020, CIKM.

[3]  Frederic Sala,et al.  Learning Mixed-Curvature Representations in Product Spaces , 2018, ICLR.

[4]  Michael M. Bronstein,et al.  MOTIFNET: A MOTIF-BASED GRAPH CONVOLUTIONAL NETWORK FOR DIRECTED GRAPHS , 2018, 2018 IEEE Data Science Workshop (DSW).

[5]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.

[6]  Lise Getoor,et al.  Query-driven Active Surveying for Collective Classification , 2012 .

[7]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[8]  Iraj Saniee,et al.  Large-scale curvature of networks. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[9]  Philip S. Yu,et al.  Lime: Low-Cost and Incremental Learning for Dynamic Heterogeneous Information Networks , 2021, IEEE Transactions on Computers.

[10]  Gary Bécigneul,et al.  Poincaré GloVe: Hyperbolic Word Embeddings , 2018, ICLR.

[11]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[12]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[13]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[14]  Douwe Kiela,et al.  Hyperbolic Graph Neural Networks , 2019, NeurIPS.

[15]  Lei Ren,et al.  A Data-Driven Approach of Product Quality Prediction for Complex Production Systems , 2021, IEEE Transactions on Industrial Informatics.

[16]  Georgios B. Giannakis,et al.  Deep Reinforcement Learning for Adaptive Caching in Hierarchical Content Delivery Networks , 2019, IEEE Transactions on Cognitive Communications and Networking.

[17]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[18]  Jure Leskovec,et al.  Hyperbolic Graph Convolutional Neural Networks , 2019, NeurIPS.

[19]  Edmond A. Jonckheere,et al.  Scaled Gromov hyperbolic graphs , 2008, J. Graph Theory.

[20]  Yilun Jin,et al.  GraLSP: Graph Neural Networks with Local Structural Patterns , 2019, AAAI.

[21]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[22]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[23]  Thomas Hofmann,et al.  Hyperbolic Neural Networks , 2018, NeurIPS.

[24]  Ujjwal Maulik,et al.  Advanced Methods for Knowledge Discovery from Complex Data , 2005 .

[25]  Timothy M. Hospedales,et al.  Multi-relational Poincaré Graph Embeddings , 2019, NeurIPS.

[26]  Jie Xu,et al.  Hawk: Rapid Android Malware Detection Through Heterogeneous Graph Attention Networks , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[27]  W. Floyd,et al.  HYPERBOLIC GEOMETRY , 1996 .

[28]  Blair D. Sullivan,et al.  Tree-Like Structure in Large Social and Information Networks , 2013, 2013 IEEE 13th International Conference on Data Mining.

[29]  Philip S. Yu,et al.  Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphs , 2021, AAAI.

[30]  Philip S. Yu,et al.  Higher-Order Attribute-Enhancing Heterogeneous Graph Neural Networks , 2021, IEEE Transactions on Knowledge and Data Engineering.

[31]  Hao Peng,et al.  SUGAR: Subgraph Neural Network with Reinforcement Pooling and Self-Supervised Mutual Information Mechanism , 2021, WWW.

[32]  W. Stadler A survey of multicriteria optimization or the vector maximum problem, part I: 1776–1960 , 1979 .

[33]  Damian Szklarczyk,et al.  The STRING database in 2017: quality-controlled protein–protein association networks, made broadly accessible , 2016, Nucleic Acids Res..

[34]  Marián Boguñá,et al.  Popularity versus similarity in growing networks , 2011, Nature.

[35]  Renquan Lu,et al.  A Wide-Deep-Sequence Model-Based Quality Prediction Method in Industrial Process Analysis , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Philip S. Yu,et al.  Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks , 2021, ACM Trans. Inf. Syst..

[37]  Octavian-Eugen Ganea,et al.  Constant Curvature Graph Convolutional Networks , 2019, ICML.

[38]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.