On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs

Graph neural networks (GNNs) have been widely used in various graph-related problems such as node classification and graph classification, where the superior performance is mainly established when natural node features are available. However, it is not well understood how GNNs work without natural node features, especially regarding the various ways to construct artificial ones. In this paper, we point out the two types of artificial node features, i.e., positional and structural node features, and provide insights on why each of them is more appropriate for certain tasks, i.e., positional node classification, structural node classification, and graph classification. Extensive experimental results on 10 benchmark datasets validate our insights, thus leading to a practical guideline on the choices between different artificial node features for GNNs on non-attributed graphs. The code is available at https://github.com/zjzijielu/gnn-exp/. ACM Reference Format: Hejie Cui, Zijie Lu, Pan Li, Carl Yang. 2021. On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs. In Proceedings of Workshop of Deep Learning on Graphs: Methods and Applications, The 27th International ACM SIGKDD Conference on Knowledge Discovery and Data Mining (DLG-KDD’21). ACM, New York, NY, USA, 5 pages.

[1]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[2]  Yihao Fang,et al.  Optimization of Graph Neural Networks with Natural Gradient Descent , 2020, 2020 IEEE International Conference on Big Data (Big Data).

[3]  Yilin Zhang,et al.  Understanding Regularized Spectral Clustering via Graph Conductance , 2018, NeurIPS.

[4]  J. Leskovec,et al.  Design Space for Graph Neural Networks , 2020, NeurIPS.

[5]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[6]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[7]  Danai Koutra,et al.  RolX: structural role extraction & mining in large graphs , 2012, KDD.

[8]  Haoteng Yin,et al.  Revisit graph neural networks and distance encoding in a practical view , 2020, ArXiv.

[9]  L. Akoglu,et al.  Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs , 2020, NeurIPS.

[10]  Marc-Alexandre Côté,et al.  Graph Policy Network for Transferable Active Learning on Graphs , 2020, NeurIPS.

[11]  Jure Leskovec,et al.  Graph Information Bottleneck , 2020, NeurIPS.

[12]  Hisashi Kashima,et al.  Random Features Strengthen Graph Neural Networks , 2020, SDM.

[13]  Jure Leskovec,et al.  Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning , 2020, NeurIPS.

[14]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[15]  A. Micheli,et al.  A Fair Comparison of Graph Neural Networks for Graph Classification , 2019, ICLR.

[16]  Ming Li,et al.  Path integral based convolution and pooling for graph neural networks , 2020, NeurIPS.

[17]  Jean-Louis Reymond,et al.  Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17 , 2012, J. Chem. Inf. Model..

[18]  Jonathan M. Gomes-Selman,et al.  Identity-aware Graph Neural Networks , 2021, AAAI.

[19]  Balasubramaniam Srinivasan,et al.  On the Equivalence between Positional Node Embeddings and Structural Graph Representations , 2019, ICLR.

[20]  Jure Leskovec,et al.  MultiSage: Empowering GCN with Contextualized Multi-Embeddings on Web-Scale Multipartite Networks , 2020, KDD.

[21]  Joan Bruna,et al.  Community Detection with Graph Neural Networks , 2017, 1705.08415.

[22]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[23]  Wenhan Shi,et al.  Conditional Structure Generation through Graph Variational Generative Adversarial Nets , 2019, NeurIPS.

[24]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[25]  Jure Leskovec,et al.  Position-aware Graph Neural Networks , 2019, ICML.

[26]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[27]  Carl Yang,et al.  Co-Embedding Network Nodes and Hierarchical Labels with Taxonomy Based Generative Adversarial Networks , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[28]  Yizhou Sun,et al.  Heterogeneous Network Representation Learning: A Unified Framework With Survey and Benchmark , 2020, IEEE Transactions on Knowledge and Data Engineering.

[29]  Siheng Chen,et al.  Learning on Attribute-Missing Graphs , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Fan Chung Graham,et al.  Spectral Clustering of Graphs with General Degrees in the Extended Planted Partition Model , 2012, COLT.

[31]  Yinglong Xia,et al.  Revisiting Graph Neural Networks for Link Prediction , 2020, ArXiv.

[32]  Jiawei Han,et al.  Relation Learning on Social Networks with Multi-Modal Graph Edge Variational Autoencoders , 2019, WSDM.

[33]  Lifang He,et al.  Graph Entropy Guided Node Embedding Dimension Selection for Graph Neural Networks , 2021, IJCAI.

[34]  Tsuyoshi Murata,et al.  Graph Convolutional Networks for Graphs Containing Missing Features , 2020, Future Gener. Comput. Syst..

[35]  Pavlo O. Dral,et al.  Quantum chemistry structures and properties of 134 kilo molecules , 2014, Scientific Data.

[36]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[37]  Davide Bacciu,et al.  Contextual Graph Markov Model: A Deep and Generative Approach to Graph Processing , 2018, ICML.

[38]  Ivor W. Tsang,et al.  Graph Cross Networks with Vertex Infomax Pooling , 2020, NeurIPS.

[39]  Thomas Lukasiewicz,et al.  The Surprising Power of Graph Neural Networks with Random Node Initialization , 2020, IJCAI.

[40]  Chen Cai,et al.  A simple yet effective baseline for non-attribute graph classification , 2018, ArXiv.

[41]  Hans-Peter Kriegel,et al.  Protein function prediction via graph kernels , 2005, ISMB.

[42]  Christos Faloutsos,et al.  It's who you know: graph mining using recursive structural features , 2011, KDD.

[43]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[44]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[45]  Samuel Kaski,et al.  Rethinking pooling in graph neural networks , 2020, NeurIPS.

[46]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[47]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[48]  A. Debnath,et al.  Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. , 1991, Journal of medicinal chemistry.

[49]  Lise Getoor,et al.  Query-driven Active Surveying for Collective Classification , 2012 .

[50]  Qian Huang,et al.  Combining Label Propagation and Simple Models Out-performs Graph Neural Networks , 2020, ICLR.

[51]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.