SIMGA: A Simple and Effective Heterophilous Graph Neural Network with Efficient Global Aggregation

Graph neural networks (GNNs) realize great success in graph learning but suffer from performance loss when meeting heterophily, i.e. neighboring nodes are dissimilar, due to their local and uniform aggregation. Existing attempts in incoorporating global aggregation for heterophilous GNNs usually require iteratively maintaining and updating full-graph information, which entails $\mathcal{O}(n^2)$ computation efficiency for a graph with $n$ nodes, leading to weak scalability to large graphs. In this paper, we propose SIMGA, a GNN structure integrating SimRank structural similarity measurement as global aggregation. The design of SIMGA is simple, yet it leads to promising results in both efficiency and effectiveness. The simplicity of SIMGA makes it the first heterophilous GNN model that can achieve a propagation efficiency near-linear to $n$. We theoretically demonstrate its effectiveness by treating SimRank as a new interpretation of GNN and prove that the aggregated node representation matrix has expected grouping effect. The performances of SIMGA are evaluated with 11 baselines on 12 benchmark datasets, usually achieving superior accuracy compared with the state-of-the-art models. Efficiency study reveals that SIMGA is up to 5$\times$ faster than the state-of-the-art method on the largest heterophily dataset pokec with over 30 million edges.

[1]  Siqiang Luo,et al.  Massively Parallel Single-Source SimRanks in o(log n) Rounds , 2023, ArXiv.

[2]  Linhao Luo,et al.  GSim: A Graph Neural Network based Relevance Measure for Heterogeneous Graphs , 2022, IEEE Transactions on Knowledge and Data Engineering.

[3]  Xiang Li,et al.  SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization , 2022, Proceedings of the VLDB Endowment.

[4]  Xiang Li,et al.  Finding Global Homophily in Graph Neural Networks When Meeting Heterophily , 2022, ICML.

[5]  Jintang Li,et al.  Spiking Graph Convolutional Networks , 2022, IJCAI.

[6]  Dongxiao He,et al.  Graph Neural Networks Beyond Compromise Between Attribute and Topology , 2022, WWW.

[7]  Kai Siong Yow,et al.  Learning-Based Approaches for Graph Problems: A Survey , 2022, ArXiv.

[8]  Philip S. Yu,et al.  Graph Neural Networks for Graphs with Heterophily: A Survey , 2022, ArXiv.

[9]  Yunhai Tong,et al.  Graph Pointer Neural Networks , 2021, AAAI.

[10]  Kevin Swersky,et al.  Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks , 2021, 2022 IEEE International Conference on Data Mining (ICDM).

[11]  Shuiwang Ji,et al.  Non-Local Graph Neural Networks , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Omkar Bhalerao,et al.  Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods , 2021, NeurIPS.

[13]  Jennifer Neville,et al.  Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patterns , 2021, KDD.

[14]  Ye Yuan,et al.  Approximate Graph Propagation , 2021, KDD.

[15]  Peng Cui,et al.  Interpreting and Unifying Graph Neural Networks with An Optimization Framework , 2021, WWW.

[16]  Olgica Milenkovic,et al.  Adaptive Universal Generalized PageRank Graph Neural Network , 2020, ICLR.

[17]  Rik Sarkar,et al.  Multi-scale Attributed Node Embedding , 2019, J. Complex Networks.

[18]  Jiawei Han,et al.  Universal Graph Convolutional Networks , 2021, NeurIPS.

[19]  Yaliang Li,et al.  Scalable Graph Neural Networks via Bidirectional Propagation , 2020, NeurIPS.

[20]  Weining Qian,et al.  Fusing Global Domain Information and Local Semantic Information to Classify Financial Documents , 2020, CIKM.

[21]  Yaliang Li,et al.  Simple and Deep Graph Convolutional Networks , 2020, ICML.

[22]  Aleksandar Bojchevski,et al.  Scaling Graph Neural Networks with Approximate PageRank , 2020, KDD.

[23]  L. Akoglu,et al.  Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs , 2020, NeurIPS.

[24]  Martin Ester,et al.  CAST: A Correlation-based Adaptive Spectral Clustering Algorithm on Multi-scale Data , 2020, KDD.

[25]  Kevin Chen-Chuan Chang,et al.  Geom-GCN: Geometric Graph Convolutional Networks , 2020, ICLR.

[26]  Stephan Günnemann,et al.  Diffusion Improves Graph Learning , 2019, NeurIPS.

[27]  Kristina Lerman,et al.  MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing , 2019, ICML.

[28]  Alexander A. Alemi,et al.  On the Use of ArXiv as a Dataset , 2019, ArXiv.

[29]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[30]  Stephan Günnemann,et al.  Predict then Propagate: Graph Neural Networks meet Personalized PageRank , 2018, ICLR.

[31]  Yaron Lipman,et al.  Invariant and Equivariant Graph Networks , 2018, ICLR.

[32]  Ken-ichi Kawarabayashi,et al.  Representation Learning on Graphs with Jumping Knowledge Networks , 2018, ICML.

[33]  Yue Wang,et al.  Efficient SimRank Tracking in Dynamic Graphs , 2018, 2018 IEEE 34th International Conference on Data Engineering (ICDE).

[34]  Stephan Günnemann,et al.  Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking , 2017, ICLR.

[35]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[36]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[37]  Guoliang Li,et al.  Efficient top-K SimRank-based similarity join , 2014, Proc. VLDB Endow..

[38]  Jure Leskovec,et al.  {SNAP Datasets}: {Stanford} Large Network Dataset Collection , 2014 .

[39]  Lise Getoor,et al.  Query-driven Active Surveying for Collective Classification , 2012 .

[40]  W. Kittisupamongkol.  Two sides of the same coin? , 2010, Singapore medical journal.

[41]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[42]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[43]  Jennifer Widom,et al.  SimRank: a measure of structural-context similarity , 2002, KDD.