Learning Invariant Graph Representations for Out-of-Distribution Generalization (Appendix)

N The number of graphs in the dataset G,Y The graph space and the label space G,Y A random variable of graph and label G,Y An instance of graph and label GI = Φ(G) An instance of the invariant subgraph and the invariant subgraph generator Φ∗ The optimal invariant subgraph generator GV = G\GI An instance of the variant subgraph AI/AV The adjacency matrix of the invariant/variant subgraph ZI/ZV The node-level representations of the invariant/variant subgraph hI/hV The graph-level representations of the invariant/variant subgraph E/Etr A random variable on indices of all/training environments Einfer A random variable on indices of the inferred environments e An instance of environment f The predictor from G to Y w The classifier from R to Y h The representation learning function from G to R g The representation learning function for invariant subgraphs IE The invariant subgraph generator set with respect to E l The loss function

[1]  Pan Li,et al.  Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism , 2022, ICML.

[2]  Xiangnan He,et al.  Discovering Invariant Rationales for Graph Neural Networks , 2022, ICLR.

[3]  Shuiwang Ji,et al.  Explainability in Graph Neural Networks: A Taxonomic Survey , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  J. Leskovec,et al.  Open Graph Benchmark: Datasets for Machine Learning on Graphs , 2020, NeurIPS.

[5]  Aaron C. Courville,et al.  Out-of-Distribution Generalization via Risk Extrapolation (REx) , 2020, ICML.

[6]  Tatsunori B. Hashimoto,et al.  Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization , 2019, ArXiv.

[7]  P. Talukdar,et al.  ASAP: Adaptive Structure Aware Pooling for Learning Hierarchical Graph Representations , 2019, AAAI.

[8]  David Lopez-Paz,et al.  Invariant Risk Minimization , 2019, ArXiv.

[9]  Shuiwang Ji,et al.  Graph U-Nets , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Mohamed R. Amer,et al.  Understanding Attention and Generalization in Graph Neural Networks , 2019, NeurIPS.

[11]  Jaewoo Kang,et al.  Self-Attention Graph Pooling , 2019, ICML.

[12]  J. Leskovec,et al.  GNNExplainer: Generating Explanations for Graph Neural Networks , 2019, NeurIPS.

[13]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[14]  Lorenzo Livi,et al.  Graph Neural Networks With Convolutional ARMA Filters , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Martin Grohe,et al.  Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks , 2018, AAAI.

[16]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[17]  Abien Fred Agarap.  Deep Learning using Rectified Linear Units (ReLU) , 2018, ArXiv.

[18]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[19]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[20]  Bernhard Schölkopf,et al.  Invariant Models for Causal Transfer Learning , 2015, J. Mach. Learn. Res..

[21]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Polina Golland,et al.  Convex Clustering with Exemplar-Based Models , 2007, NIPS.

[23]  Thomas M. Cover,et al.  Network Information Theory , 2001 .

[24]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[25]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[26]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.