Topological Graph Neural Networks

Graph neural networks (GNNs) are a powerful architecture for tackling graph learning tasks, yet have been shown to be oblivious to eminent substructures, such as cycles. We present TOGL, a novel layer that incorporates global topological information of a graph using persistent homology. TOGL can be easily integrated into any type of GNN and is strictly more expressive in terms of the Weisfeiler–Lehman test of isomorphism. Augmenting GNNs with our layer leads to beneficial predictive performance, both on synthetic data sets, which can be trivially classified by humans but not by ordinary GNNs, and on real-world data.

[1]  A. A. LEMAN,et al.  THE REDUCTION OF A GRAPH TO CANONICAL FORM AND THE ALGEBRA WHICH APPEARS THEREIN , 2018 .

[2]  Martin Fürer,et al.  On the Combinatorial Power of the Weisfeiler-Lehman Algorithm , 2017, CIAC.

[3]  R. Ho.  Algebraic Topology , 2022 .

[4]  Karsten M. Borgwardt,et al.  Topological Autoencoders , 2019, ICML.

[5]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Xavier Bresson,et al.  Residual Gated Graph ConvNets , 2017, ArXiv.

[7]  Paul Erdös,et al.  Random Graph Isomorphism , 1980, SIAM J. Comput..

[8]  Karsten M. Borgwardt,et al.  A Persistent Weisfeiler-Lehman Procedure for Graph Classification , 2019, ICML.

[9]  Kristian Kersting,et al.  TUDataset: A collection of benchmark datasets for learning with graphs , 2020, ArXiv.

[10]  Marc Niethammer,et al.  Graph Filtration Learning , 2019, ICML.

[11]  Xavier Bresson,et al.  Benchmarking Graph Neural Networks , 2020, ArXiv.

[12]  Don Sheehy,et al.  Linear-Size Approximations to the Vietoris–Rips Filtration , 2012, Discrete & Computational Geometry.

[13]  Katherine A. Yelick,et al.  PersGNN: Applying Topological Data Analysis and Geometric Deep Learning to Structure-Based Protein Function Prediction , 2020, ArXiv.

[14]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[15]  Karsten M. Borgwardt,et al.  Fast subtree kernels on graphs , 2009, NIPS.

[16]  David Cohen-Steiner,et al.  Extending Persistence Using Poincaré and Lefschetz Duality , 2009, Found. Comput. Math..

[17]  Nils M. Kriege,et al.  On Valid Optimal Assignment Kernels and Applications to Graph Classification , 2016, NIPS.

[18]  Michael Moor,et al.  A Survey of Topological Machine Learning Methods , 2021, Frontiers in Artificial Intelligence.

[19]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[20]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[21]  Nicholas J. Cavanna,et al.  Visualizing Sparse Filtrations , 2015, Symposium on Computational Geometry.

[22]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[23]  Herbert Edelsbrunner,et al.  Computational Topology - an Introduction , 2009 .

[24]  Stefanos Zafeiriou,et al.  Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting , 2020, ArXiv.

[25]  Michalis Vazirgiannis,et al.  Graph Kernels: A Survey , 2019, J. Artif. Intell. Res..

[26]  Andreas Uhl,et al.  Deep Learning with Topological Signatures , 2017, NIPS.

[27]  Larry Wasserman,et al.  PLLay: Efficient Topological Layer based on Persistent Landscapes , 2020, NeurIPS.

[28]  Joan Bruna,et al.  On the equivalence between graph isomorphism testing and function approximation with GNNs , 2019, NeurIPS.

[29]  Oleg Verbitsky,et al.  On Weisfeiler-Leman Invariance: Subgraph Counts and Related Graph Properties , 2018, FCT.

[30]  S. A. Barannikov,et al.  The framed Morse complex and its invariants , 1994 .

[31]  Primoz Skraba,et al.  Zigzag persistent homology in matrix multiplication time , 2011, SoCG '11.

[32]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[33]  Nicholas J. Cavanna,et al.  A Geometric Perspective on Sparse Filtrations , 2015, CCCG.

[34]  Herbert Edelsbrunner,et al.  Topological Persistence and Simplification , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[35]  Paul Bendich,et al.  Stabilizing the unstable output of persistent homology computations , 2015, Journal of Applied and Computational Topology.

[36]  Patrizio Frosini,et al.  On the use of size functions for shape analysis , 1993, [1993] Proceedings IEEE Workshop on Qualitative Vision.

[37]  Christian Bock,et al.  Uncovering the Topology of Time-Varying fMRI Data using Cubical Persistence , 2020, NeurIPS.

[38]  Joan Bruna,et al.  Can graph neural networks count substructures? , 2020, NeurIPS.

[39]  Chao Chen,et al.  Persistence Enhanced Graph Neural Network , 2020, AISTATS.

[40]  Martin Grohe,et al.  Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks , 2018, AAAI.

[41]  Brendan D. McKay,et al.  Classification of regular two-graphs on 36 and 38 vertices , 2001, Australas. J Comb..

[42]  Marc Niethammer,et al.  Learning Representations of Persistence Barcodes , 2019, J. Mach. Learn. Res..

[43]  Qi Zhao,et al.  Learning metrics for persistence-based summaries and applications for graph classification , 2019, NeurIPS.

[44]  Jie Zhou,et al.  Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View , 2020, AAAI.

[45]  Marcio Gameiro,et al.  Continuation of Point Clouds via Persistence Diagrams , 2015, ArXiv.

[46]  Mathieu Carrière,et al.  PersLay: A Neural Network Layer for Persistence Diagrams and New Graph Topological Signatures , 2020, AISTATS.

[47]  Frédéric Chazal,et al.  Optimizing persistent homology based functions , 2020, ICML.

[48]  Maks Ovsjanikov,et al.  Topological Function Optimization for Continuous Shape Matching , 2018, Comput. Graph. Forum.

[49]  Afra Zomorodian,et al.  Computing Multidimensional Persistence , 2009, J. Comput. Geom..

[50]  Yaron Lipman,et al.  Provably Powerful Graph Networks , 2019, NeurIPS.

[51]  Nils M. Kriege,et al.  A survey on graph kernels , 2019, Applied Network Science.

[52]  Afra Zomorodian,et al.  Fast construction of the Vietoris-Rips complex , 2010, Comput. Graph..

[53]  László Babai,et al.  Canonical labelling of graphs in linear average time , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[54]  Guido Mont'ufar,et al.  Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks , 2021, ICML.