Ego-Based Entropy Measures for Structural Representations on Graphs

Machine learning on graph-structured data has attracted high research interest due to the emergence of Graph Neural Networks (GNNs). Most of the proposed GNNs are based on the node homophily, i.e neighboring nodes share similar characteristics. However, in many complex networks, nodes that lie to distant parts of the graph share structurally equivalent characteristics and exhibit similar roles (e.g chemical properties of distant atoms in a molecule, type of social network users). A growing literature proposed representations that identify structurally equivalent nodes. However, most of the existing methods require high time and space complexity. In this paper, we propose VNEstruct, a simple approach, based on entropy measures of the neighborhood’s topology, for generating low-dimensional structural representations, that is time- efficient and robust to graph perturbations. Empirically, we observe that VNEstruct exhibits robustness on structural role identification tasks. Moreover, VNEstruct can achieve state- of-the-art performance on graph classification, without incorporating the graph structure information in the optimization, in contrast to GNN competitors.

[1]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[2]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[3]  Ludovic Dos Santos,et al.  Coloring graph neural networks for node disambiguation , 2019, IJCAI.

[4]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[5]  Andrea Torsello,et al.  On the von Neumann entropy of graphs , 2018, J. Complex Networks.

[6]  Yizhou Sun,et al.  Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification , 2019, ArXiv.

[7]  S. Severini,et al.  The Laplacian of a Graph as a Density Matrix: A Basic Combinatorial Approach to Separability of Mixed States , 2004, quant-ph/0406165.

[8]  Simone Severini,et al.  Quantifying Complexity in Networks: The von Neumann Entropy , 2009, Int. J. Agent Technol. Syst..

[9]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[10]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[11]  R. Mises,et al.  Praktische Verfahren der Gleichungsauflösung . , 1929 .

[12]  Leman Akoglu,et al.  PairNorm: Tackling Oversmoothing in GNNs , 2020, ICLR.

[13]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[14]  Sanjay Joshua Swamidass,et al.  Deep learning long-range information in undirected graphs with wave networks , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[15]  K. Audenaert A sharp continuity estimate for the von Neumann entropy , 2006, quant-ph/0610146.

[16]  Lihui Chen,et al.  Capsule Graph Neural Network , 2018, ICLR.

[17]  Michalis Vazirgiannis,et al.  k-hop Graph Neural Networks , 2019, Neural Networks.

[18]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[19]  Danai Koutra,et al.  RolX: structural role extraction & mining in large graphs , 2012, KDD.

[20]  Angsheng Li,et al.  Structural Information and Dynamical Complexity of Networks , 2016, IEEE Transactions on Information Theory.

[21]  Sijia Liu,et al.  Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications , 2018, ICML.

[22]  G. Bianconi,et al.  Shannon and von Neumann entropy of random networks with heterogeneous expected degree. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[23]  Yuanming Shi,et al.  Fast computation of von Neumann entropy for large-scale graphs via quadratic approximations , 2018 .

[24]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[25]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[26]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[27]  Philip S. Yu,et al.  Deep Recursive Network Embedding with Regular Equivalence , 2018, KDD.

[28]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.