On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs

Graph neural networks (GNNs) have been widely used in various graph-related problems such as node classification and graph classification, where the superior performance is mainly established when natural node features are available. However, it is not well understood how GNNs work without natural node features, especially regarding the various ways to construct artificial ones. In this paper, we point out the two types of artificial node features, i.e., positional and structural node features, and provide insights on why each of them is more appropriate for certain tasks, i.e., positional node classification, structural node classification, and graph classification. Extensive experimental results on 10 benchmark datasets validate our insights, thus leading to a practical guideline on the choices between different artificial node features for GNNs on non-attributed graphs. The code is available at https://github.com/zjzijielu/gnn-positional-structural-node-features.

[1]  Carl Yang,et al.  Interpretable Graph Neural Networks for Connectome-Based Brain Disorder Analysis , 2022, MICCAI.

[2]  Carl Yang,et al.  Data-Efficient Brain Connectome Analysis via Multi-Task Meta-Learning , 2022, KDD.

[3]  Carl Yang,et al.  FBNETGEN: Task-aware GNN-based fMRI Analysis via Functional Brain Network Generation , 2022, MIDL.

[4]  Yuan Fang,et al.  On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks , 2022, WWW.

[5]  Carl Yang,et al.  BrainGB: A Benchmark for Brain Network Analysis With Graph Neural Networks , 2022, IEEE Transactions on Medical Imaging.

[6]  Xiaojie Guo,et al.  Deep Generative Model for Periodic Graphs , 2022, Neural Information Processing Systems.

[7]  Carl Yang,et al.  Zero-Shot Scene Graph Relation Prediction through Commonsense Knowledge Integration , 2021, ECML/PKDD.

[8]  Steven C. H. Hoi,et al.  Relative and Absolute Location Embedding for Few-Shot Node Classification on Graph , 2021, AAAI.

[9]  Lifang He,et al.  Graph Entropy Guided Node Embedding Dimension Selection for Graph Neural Networks , 2021, IJCAI.

[10]  Jonathan M. Gomes-Selman,et al.  Identity-aware Graph Neural Networks , 2021, AAAI.

[11]  Haoteng Yin,et al.  Revisit graph neural networks and distance encoding in a practical view , 2020, ArXiv.

[12]  J. Leskovec,et al.  Design Space for Graph Neural Networks , 2020, NeurIPS.

[13]  Carl Yang,et al.  Co-Embedding Network Nodes and Hierarchical Labels with Taxonomy Based Generative Adversarial Networks , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[14]  Yinglong Xia,et al.  Revisiting Graph Neural Networks for Link Prediction , 2020, ArXiv.

[15]  Qian Huang,et al.  Combining Label Propagation and Simple Models Out-performs Graph Neural Networks , 2020, ICLR.

[16]  Jure Leskovec,et al.  Graph Information Bottleneck , 2020, NeurIPS.

[17]  Samuel Kaski,et al.  Rethinking pooling in graph neural networks , 2020, NeurIPS.

[18]  I. Tsang,et al.  Graph Cross Networks with Vertex Infomax Pooling , 2020, NeurIPS.

[19]  Thomas Lukasiewicz,et al.  The Surprising Power of Graph Neural Networks with Random Node Initialization , 2020, IJCAI.

[20]  Siheng Chen,et al.  Learning on Attribute-Missing Graphs , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  J. Leskovec,et al.  Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning , 2020, NeurIPS.

[22]  Yihao Fang,et al.  Optimization of Graph Neural Networks with Natural Gradient Descent , 2020, 2020 IEEE International Conference on Big Data (Big Data).

[23]  Kristian Kersting,et al.  TUDataset: A collection of benchmark datasets for learning with graphs , 2020, ArXiv.

[24]  T. Murata,et al.  Graph Convolutional Networks for Graphs Containing Missing Features , 2020, Future Gener. Comput. Syst..

[25]  Jure Leskovec,et al.  MultiSage: Empowering GCN with Contextualized Multi-Embeddings on Web-Scale Multipartite Networks , 2020, KDD.

[26]  Ming Li,et al.  Path integral based convolution and pooling for graph neural networks , 2020, NeurIPS.

[27]  L. Akoglu,et al.  Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs , 2020, NeurIPS.

[28]  Marc-Alexandre Côté,et al.  Graph Policy Network for Transferable Active Learning on Graphs , 2020, NeurIPS.

[29]  Christopher Ré,et al.  Machine Learning on Graphs: A Model and Comprehensive Taxonomy , 2020, J. Mach. Learn. Res..

[30]  Carl Yang,et al.  Heterogeneous Network Representation Learning: A Unified Framework With Survey and Benchmark , 2020, IEEE Transactions on Knowledge and Data Engineering.

[31]  Hisashi Kashima,et al.  Random Features Strengthen Graph Neural Networks , 2020, SDM.

[32]  A. Micheli,et al.  A Fair Comparison of Graph Neural Networks for Graph Classification , 2019, ICLR.

[33]  Karl Aberer,et al.  On Node Features for Graph Neural Networks , 2019, ArXiv.

[34]  Jiawei Han,et al.  Relation Learning on Social Networks with Multi-Modal Graph Edge Variational Autoencoders , 2019, WSDM.

[35]  Bruno Ribeiro,et al.  On the Equivalence between Positional Node Embeddings and Structural Graph Representations , 2019, ICLR.

[36]  Jure Leskovec,et al.  Position-aware Graph Neural Networks , 2019, ICML.

[37]  Yusu Wang,et al.  A simple yet effective baseline for non-attribute graph classification , 2018, ArXiv.

[38]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[39]  Yilin Zhang,et al.  Understanding Regularized Spectral Clustering via Graph Conductance , 2018, NeurIPS.

[40]  Davide Bacciu,et al.  Contextual Graph Markov Model: A Deep and Generative Approach to Graph Processing , 2018, ICML.

[41]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[42]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[43]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[44]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[45]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[46]  Joan Bruna,et al.  Supervised Community Detection with Line Graph Neural Networks , 2017, ICLR.

[47]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[48]  Zhou Zhao,et al.  Semantic Proximity Search on Heterogeneous Graph by Proximity Embedding , 2017, AAAI Conference on Artificial Intelligence.

[49]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[50]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[51]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[52]  Pavlo O. Dral,et al.  Quantum chemistry structures and properties of 134 kilo molecules , 2014, Scientific Data.

[53]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[54]  Jean-Louis Reymond,et al.  Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17 , 2012, J. Chem. Inf. Model..

[55]  Danai Koutra,et al.  RolX: structural role extraction & mining in large graphs , 2012, KDD.

[56]  Fan Chung Graham,et al.  Spectral Clustering of Graphs with General Degrees in the Extended Planted Partition Model , 2012, COLT.

[57]  C. Faloutsos,et al.  RolX: Role Extraction and Mining in Large Networks , 2011 .

[58]  Christos Faloutsos,et al.  It's who you know: graph mining using recursive structural features , 2011, KDD.

[59]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[60]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[61]  A. Debnath,et al.  Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. , 1991, Journal of medicinal chemistry.

[62]  Xiaojie Guo,et al.  Graph Neural Networks: Graph Transformation , 2022, Graph Neural Networks: Foundations, Frontiers, and Applications.

[63]  Carl Yang,et al.  Structure-Enhanced Heterogeneous Graph Contrastive Learning , 2022, SDM.

[64]  Wenhan Shi,et al.  Conditional Structure Generation through Graph Variational Generative Adversarial Nets , 2019, NeurIPS.

[65]  Lise Getoor,et al.  Query-driven Active Surveying for Collective Classification , 2012 .

[66]  Hans-Peter Kriegel,et al.  Protein function prediction via graph kernels , 2005, ISMB.