Is Homophily a Necessity for Graph Neural Networks?

Graph neural networks (GNNs) have shown great prowess in learning representations suitable for numerous graph-based machine learning tasks. When applied to semi-supervised node classification, GNNs are widely believed to work well due to the homophily assumption ("like attracts like"), and fail to generalize to heterophilous graphs where dissimilar nodes connect. Recent works design new architectures to overcome such heterophily-related limitations, citing poor baseline performance and new architecture improvements on a few heterophilous graph benchmark datasets as evidence for this notion. In our experiments, we empirically find that standard graph convolutional networks (GCNs) can actually achieve better performance than such carefully designed methods on some commonly used heterophilous graphs. This motivates us to reconsider whether homophily is truly necessary for good GNN performance. We find that this claim is not quite true, and in fact, GCNs can achieve strong performance on heterophilous graphs under certain conditions. Our work carefully characterizes these conditions, and provides supporting theoretical understanding and empirical observations. Finally, we examine existing heterophilous graphs benchmarks and reconcile how the GCN (under)performs on them based on this understanding.

[1]  Bryan Perozzi,et al.  Synthetic Graph Generation to Benchmark Graph Learning , 2022, ArXiv.

[2]  Doina Precup,et al.  Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification? , 2021, ArXiv.

[3]  Tsuyoshi Murata,et al.  Improving Graph Neural Networks with Simple Architecture Design , 2021, ArXiv.

[4]  Neil Shah,et al.  Graph Neural Networks for Friend Ranking in Large-scale Social Platforms , 2021, WWW.

[5]  K. Fountoulakis,et al.  Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization , 2021, ICML.

[6]  Peng Cui,et al.  Interpreting and Unifying Graph Neural Networks with An Optimization Framework , 2021, WWW.

[7]  Weiwei Jiang,et al.  Graph Neural Network for Traffic Forecasting: A Survey , 2021, Expert Syst. Appl..

[8]  Austin R. Benson,et al.  A Unifying Generative Model for Graph Learning Algorithms: Label Propagation, Graph Convolutions, and Combinations , 2021, SIAM J. Math. Data Sci..

[9]  Jiliang Tang,et al.  A Unified View on Graph Neural Networks as Graph Signal Denoising , 2020, CIKM.

[10]  Ryan A. Rossi,et al.  Graph Neural Networks with Heterophily , 2020, AAAI.

[11]  Suhang Wang,et al.  Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information , 2020, WSDM.

[12]  Bryan Perozzi,et al.  Grale: Designing Networks for Graph Learning , 2020, KDD.

[13]  Yaliang Li,et al.  Simple and Deep Graph Convolutional Networks , 2020, ICML.

[14]  Jiliang Tang,et al.  Investigating and Mitigating Degree-Related Biases in Graph Convoltuional Networks , 2020, CIKM.

[15]  L. Akoglu,et al.  Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs , 2020, NeurIPS.

[16]  Olgica Milenkovic,et al.  Adaptive Universal Generalized PageRank Graph Neural Network , 2020, ICLR.

[17]  Leonardo Neves,et al.  Data Augmentation for Graph Neural Networks , 2020, AAAI.

[18]  Xianfeng Tang,et al.  Knowing your FATE: Friendship, Action and Temporal Explanations for User Engagement Prediction on Social Apps , 2020, KDD.

[19]  Hongzhi Chen,et al.  Measuring and Improving the Use of Graph Information in Graph Neural Networks , 2020, ICLR.

[20]  Kevin Chen-Chuan Chang,et al.  Geom-GCN: Geometric Graph Convolutional Networks , 2020, ICLR.

[21]  Lei Shi,et al.  Skeleton-Based Action Recognition With Directed Graph Neural Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Taiji Suzuki,et al.  Graph Neural Networks Exponentially Lose Expressive Power for Node Classification , 2019, ICLR.

[23]  A. Galstyan,et al.  MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing , 2019, ICML.

[24]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[25]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[26]  Leman Akoglu,et al.  A Quest for Structure: Jointly Learning the Graph Structure and Semi-Supervised Classification , 2018, CIKM.

[27]  Martin Grohe,et al.  Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks , 2018, AAAI.

[28]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[29]  Stephan Günnemann,et al.  Predict then Propagate: Graph Neural Networks meet Personalized PageRank , 2018, ICLR.

[30]  Andrea Montanari,et al.  Contextual Stochastic Block Models , 2018, NeurIPS.

[31]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[32]  Kevin Scaman,et al.  Lipschitz regularity of deep neural networks: analysis and efficient estimation , 2018, NeurIPS.

[33]  Jure Leskovec,et al.  Modeling polypharmacy side effects with graph convolutional networks , 2018, bioRxiv.

[34]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[35]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[36]  Khalil Sima'an,et al.  Graph Convolutional Encoders for Syntax-aware Neural Machine Translation , 2017, EMNLP.

[37]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[38]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[39]  Santo Fortunato,et al.  Community detection in networks: A user guide , 2016, ArXiv.

[40]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[41]  Vito Latora,et al.  Homophily and missing links in citation networks , 2015, EPJ Data Science.

[42]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[43]  Elisabeth R. Gerber,et al.  Political Homophily and Collaboration in Regional Planning Networks , 2013 .

[44]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[45]  M. McPherson,et al.  Birds of a Feather: Homophily in Social Networks , 2001 .

[46]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[47]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.