Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications

The von Neumann graph entropy (VNGE) facilitates measurement of information divergence and distance between graphs in a graph sequence. It has been successfully applied to various learning tasks driven by network-based data. While effective, VNGE is computationally demanding as it requires the full eigenspectrum of the graph Laplacian matrix. In this paper, we propose a new computational framework, Fast Incremental von Neumann Graph EntRopy (FINGER), which approaches VNGE with a performance guarantee. FINGER reduces the cubic complexity of VNGE to linear complexity in the number of nodes and edges, and thus enables online computation based on incremental graph changes. We also show asymptotic equivalence of FINGER to the exact VNGE, and derive its approximation error bounds. Based on FINGER, we propose efficient algorithms for computing Jensen-Shannon distance between graphs. Our experimental results on different random graph models demonstrate the computational efficiency and the asymptotic equivalence of FINGER. In addition, we apply FINGER to two real-world applications and one synthesized anomaly detection dataset, and corroborate its superior performance over seven baseline graph similarity methods.

[1]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[2]  T. Sørensen,et al.  A method of establishing group of equal amplitude in plant sociobiology based on similarity of species content and its application to analyses of the vegetation on Danish commons , 1948 .

[3]  M. Fiedler.  Algebraic connectivity of graphs , 1973 .

[4]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[5]  T. D. Morley,et al.  Eigenvalues of the Laplacian of a graph , 1985 .

[6]  S. Tapscott,et al.  Activation of muscle-specific genes in pigment, nerve, fat, liver, and fibroblast cell lines by forced expression of MyoD. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Harold Weintraub,et al.  The MyoD family and myogenesis: Redundancy, networks, and thresholds , 1993, Cell.

[8]  Gábor Simonyi,et al.  Graph entropy: A survey , 1993, Combinatorial Optimization.

[9]  R. Merris.  Laplacian matrices of graphs: a survey , 1994 .

[10]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[12]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[13]  Jack Dongarra,et al.  Templates for the Solution of Algebraic Eigenvalue Problems , 2000, Software, environments, tools.

[14]  K. Goh,et al.  Spectra and eigenvectors of scale-free networks. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Dominik Endres,et al.  A new metric for probability distributions , 2003, IEEE Transactions on Information Theory.

[16]  D. A. Edwards.  The mathematical foundations of quantum mechanics , 1979, Synthese.

[17]  Jafar Adibi,et al.  Discovering important nodes through graph entropy the case of Enron email database , 2005, LinkKDD '05.

[18]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[19]  F. Chung.  Laplacians and the Cheeger Inequality for Directed Graphs , 2005 .

[20]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[21]  S. Severini,et al.  The Laplacian of a Graph as a Density Matrix: A Basic Combinatorial Approach to Separability of Mixed States , 2004, quant-ph/0406165.

[22]  W. Wallis,et al.  A Graph-Theoretic Approach to Enterprise Network Dynamics , 2006 .

[23]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[24]  P. Harremoes,et al.  Properties of Classical and Quantum Jensen-Shannon Divergence , 2008, 0806.4472.

[25]  Ping Zhu,et al.  A study of graph spectra for comparing graphs and trees , 2008, Pattern Recognit..

[26]  Simone Severini,et al.  The von Neumann Entropy of Networks , 2008, 0812.2597.

[27]  F. Pazos,et al.  Reactome Array: Forging a Link Between Metabolome and Genome , 2009, Science.

[28]  Peter Druschel,et al.  Online social networks: measurement, analysis, and applications to distributed information systems , 2009 .

[29]  Ginestra Bianconi,et al.  Entropy measures for networks: toward an information theory of complex topologies. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[30]  Simone Severini,et al.  Quantifying Complexity in Networks: The von Neumann Entropy , 2009, Int. J. Agent Technol. Syst..

[31]  Hector Garcia-Molina,et al.  Web graph similarity for anomaly detection , 2010, Journal of Internet Services and Applications.

[32]  Simone Severini,et al.  A note on the von Neumann entropy of random graphs , 2010 .

[33]  Piet Van Mieghem,et al.  Graph Spectra for Complex Networks , 2010 .

[34]  G. Bianconi,et al.  Shannon and von Neumann entropy of random networks with heterogeneous expected degree. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[35]  Edwin R. Hancock,et al.  Graph characterizations from von Neumann entropy , 2012, Pattern Recognit. Lett..

[36]  Feiping Nie,et al.  Forging The Graphs: A Low Rank and Positive Semidefinite Graph Learning Approach , 2012, NIPS.

[37]  Akshay Krishnamurthy,et al.  Near-optimal Anomaly Detection in Graphs using Lovasz Extended Scan Statistic , 2013, NIPS.

[38]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[39]  Alfred O. Hero,et al.  Node removal vulnerability of the largest component of a network , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[40]  Steffen Staab,et al.  Structural Dynamics of Knowledge Networks , 2013, ICWSM.

[41]  Chiranjib Bhattacharyya,et al.  Learning on graphs using Orthonormal Representation is Statistically Consistent , 2014, NIPS.

[42]  Richard C. Wilson,et al.  Approximate von Neumann entropy for directed graphs. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  Edwin R. Hancock,et al.  Depth-based complexity traces of graphs , 2014, Pattern Recognit..

[44]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[45]  Steve Harenberg,et al.  Anomaly detection in dynamic networks: a survey , 2015 .

[46]  S. V. N. Vishwanathan,et al.  A Structural Smoothing Framework For Robust Graph Comparison , 2015, NIPS.

[47]  Vito Latora,et al.  Structural reducibility of multilayer networks , 2015, Nature Communications.

[48]  Vassilis Kalofolias,et al.  How to Learn a Graph from Smooth Signals , 2016, AISTATS.

[49]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[50]  Angsheng Li,et al.  Structural Information and Dynamical Complexity of Networks , 2016, IEEE Transactions on Information Theory.

[51]  Aarti Singh,et al.  Graph Connectivity in Noisy Sparse Subspace Clustering , 2015, AISTATS.

[52]  Alfred O. Hero,et al.  Multi-centrality graph spectral decompositions and their application to cyber intrusion detection , 2015, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[53]  Danai Koutra,et al.  DeltaCon: Principled Massive-Graph Similarity Function with Attribution , 2016, ACM Trans. Knowl. Discov. Data.

[54]  Domitilla Del Vecchio,et al.  A Blueprint for a Synthetic Genetic Feedback Controller to Reprogram Cell Fate. , 2017, Cell systems.

[55]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[56]  Gerald A. Higgins,et al.  Genome Architecture Leads a Bifurcation in Cell Identity , 2017, bioRxiv.

[57]  Eloy Romero,et al.  PRIMME_SVDS: A High-Performance Preconditioned SVD Solver for Accurate Large-Scale Computations , 2016, SIAM J. Sci. Comput..

[58]  Darawalee Wangsa,et al.  Nucleome Analysis Reveals Structure–Function Relationships for Colon Cancer , 2017, Molecular Cancer Research.

[59]  Alfred O. Hero,et al.  Dynamic Network Analysis of the 4D Nucleome , 2018, bioRxiv.

[60]  Pierre Baldi,et al.  Genome Architecture Mediates Transcriptional Control of Human Myogenic Reprogramming , 2018, iScience.

[61]  Charu C. Aggarwal,et al.  Scalable Spectral Clustering Using Random Binning Features , 2018, KDD.

[62]  Pradeep Ravikumar,et al.  D2KE: From Distance to Kernel and Embedding , 2018, ArXiv.

[63]  Yansong Feng,et al.  Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks , 2018, ArXiv.

[64]  Dane Taylor,et al.  Network-ensemble comparisons with stochastic rewiring and von Neumann entropy , 2017, SIAM J. Appl. Math..

[65]  Zhizhen Zhao,et al.  LanczosNet: Multi-Scale Deep Graph Convolutional Networks , 2019, ICLR.