PBGAN: Path Based Graph Attention Network for Heterophily

Typical Graph Neural Networks (GNNs) have achieved noticeable performance in node classification tasks with the strategy of neighbor aggregation, which benefits from the homophily assumption of graphs (nodes with the same class tend to be connected). However, the performance of many models degrades tensely when data tend to be heterophilic. Therefore, we need to optimize the neighbor aggregation strategy to deal with these two different types of graphs. In this paper, we propose a Path Based Graph Attention Network (PBGAN) to address this issue. It utilizes structure and path information in the high-order neighborhood and employs Recurrent Neural Network (RNN) to learn attention coefficients according to the shortest paths between node pairs. This learning method makes the attention coefficients better reflect the relational importance between nodes, which is practical for processing heterophilic graphs. Empirical results demonstrate that the PBGAN achieves comparable performance in both homophilic and heterophilic datasets, proving the effectiveness of the proposed attention learning method.

[1]  Kevin Swersky,et al.  Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks , 2021, 2022 IEEE International Conference on Data Mining (ICDM).

[2]  Alice H. Oh,et al.  How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision , 2022, ICLR.

[3]  Di He,et al.  Do Transformers Really Perform Bad for Graph Representation? , 2021, ArXiv.

[4]  Wenbin Hu,et al.  Enhancing Graph Neural Networks by a High-quality Aggregation of Beneficial Information , 2021, Neural Networks.

[5]  Xiao Wang,et al.  Beyond Low-frequency Information in Graph Convolutional Networks , 2021, AAAI.

[6]  Ryan A. Rossi,et al.  Graph Neural Networks with Heterophily , 2020, AAAI.

[7]  James Cheng,et al.  Self-Enhanced GNN: Improving Graph Neural Networks Using Model Outputs , 2020, 2021 International Joint Conference on Neural Networks (IJCNN).

[8]  L. Akoglu,et al.  Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs , 2020, NeurIPS.

[9]  Jun Wang,et al.  Adaptive Structural Fingerprints for Graph Attention Networks , 2020, ICLR.

[10]  Austin R. Benson,et al.  Residual Correlation in Graph Neural Network Regression , 2020, KDD.

[11]  Kevin Chen-Chuan Chang,et al.  Geom-GCN: Geometric Graph Convolutional Networks , 2020, ICLR.

[12]  Xu Sun,et al.  Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View , 2019, AAAI.

[13]  Jure Leskovec,et al.  Improving Graph Attention Networks with Large Margin-based Constraints , 2019, ArXiv.

[14]  Dacheng Tao,et al.  SPAGAN: Shortest Path Graph Attention Network , 2019, IJCAI.

[15]  Kristina Lerman,et al.  MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing , 2019, ICML.

[16]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[17]  Ryan A. Rossi,et al.  Attention Models in Graphs , 2018, ACM Trans. Knowl. Discov. Data.

[18]  Andrew Tomkins,et al.  Graph Agreement Models for Semi-Supervised Learning , 2019, NeurIPS.

[19]  Markus Strohmaier,et al.  Homophily influences ranking of minorities in social networks , 2018, Scientific Reports.

[20]  Ryan A. Rossi,et al.  Attention Models in Graphs: A Survey , 2018 .

[21]  Hao Ma,et al.  GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs , 2018, UAI.

[22]  Chong Wang,et al.  Attention-based Graph Neural Network for Semi-supervised Learning , 2018, ArXiv.

[23]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[24]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[25]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[26]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[27]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[28]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[29]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[30]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[31]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[32]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[33]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[34]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[35]  M. McPherson,et al.  Birds of a Feather: Homophily in Social Networks , 2001 .

[36]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.