Entropic characterization and time evolution of complex networks

In this thesis, we address problems encountered in complex network analysis using graph theoretic methods. The thesis specifically centers on the challenge of how to characterize the structural properties and time evolution of graphs. We commence by providing a brief roadmap for our research in Chapter 1, followed by a review of the relevant research literature in Chapter 2. The remainder of the thesis is structured as follows. In Chapter 3, we focus on the graph entropic characterizations and explore whether the von Neumann entropy recently defined only on undirected graphs, can be extended to the domain of directed graphs. The substantial contribution involves a simplified form of the entropy which can be expressed in terms of simple graph statistics, such as graph size and vertex in-degree and out-degree. Chapter 4 further investigates the uses and applications of the von Neumann entropy in order to solve a number of network analysis and machine learning problems. The contribution in this chapter includes an entropic edge assortativity measure and an entropic graph embedding method, which are developed for both undirected and directed graphs. The next part of the thesis analyzes the time-evolving complex networks using physical and information theoretic approaches. In particular, Chapter 5 provides a thermodynamic framework for handling dynamic graphs using ideas from algebraic graph theory and statistical mechanics. This allows us to derive expressions for a number of thermodynamic functions, including energy, entropy and temperature, which are shown to be efficient in identifying abrupt structural changes and phase transitions in real-world dynamical systems. Chapter 6 develops a novel method for constructing a generative model to analyze the structure of labeled data, which provides a number of novel directions to the study of graph time-series. Finally, in Chapter 7, we provide concluding remarks and discuss the limitations of our methodologies, and point out possible future research directions.

[1]  G. Bianconi,et al.  Shannon and von Neumann entropy of random networks with heterogeneous expected degree. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[3]  Benno Schwikowski,et al.  Graph-based methods for analysing networks in cell biology , 2006, Briefings Bioinform..

[4]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[5]  S. Severini,et al.  The Laplacian of a Graph as a Density Matrix: A Basic Combinatorial Approach to Separability of Mixed States , 2004, quant-ph/0406165.

[6]  S. Rosenberg.  The Laplacian on a Riemannian Manifold: An Introduction to Analysis on Manifolds , 1997 .

[7]  Nair Maria Maia de Abreu,et al.  The characteristic polynomial of the Laplacian of graphs in (a,b)-linear classes , 2002 .

[8]  J. Crutchfield,et al.  Measures of statistical complexity: Why? , 1998 .

[9]  Davide Luciani,et al.  From networks of protein interactions to networks of functional dependencies , 2012, BMC Systems Biology.

[10]  Simone Severini,et al.  The von Neumann Entropy of Networks , 2008, 0812.2597.

[11]  Søren Riis.  Graph Entropy, Network Coding and Guessing games , 2007, ArXiv.

[12]  Massimo Piccardi,et al.  Discriminative prototype selection methods for graph embedding , 2013, Pattern Recognit..

[13]  Daniel Cremers,et al.  The wave kernel signature: A quantum mechanical approach to shape analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[14]  Shariefuddin Pirzada,et al.  Applications of graph theory , 2007 .

[15]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[16]  L. da F. Costa,et al.  Characterization of complex networks: A survey of measurements , 2005, cond-mat/0505185.

[17]  J. Geweke,et al.  Measurement of Linear Dependence and Feedback between Multiple Time Series , 1982 .

[18]  Raphael H. Heiberger,et al.  Stock network stability in times of crisis , 2014 .

[19]  Vijaya Ramachandran,et al.  The diameter of sparse random graphs , 2007, Random Struct. Algorithms.

[20]  E. M.,et al.  Statistical Mechanics , 2021, Manual for Theoretical Chemistry.

[21]  Lucas Antiqueira,et al.  Modeling connectivity in terms of network activity , 2009 .

[22]  Antje Chang,et al.  BRENDA , the enzyme database : updates and major new developments , 2003 .

[23]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[24]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[25]  Edwin R. Hancock,et al.  Generative Graph Prototypes from Information Theory , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Agata Fronczak,et al.  Thermodynamic forces, flows, and Onsager coefficients in complex networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[28]  Jens Christian Claussen,et al.  Offdiagonal complexity: A computationally quick complexity measure for graphs and networks , 2004, q-bio/0410024.

[29]  Edwin R. Hancock,et al.  Graph Kernels from the Jensen-Shannon Divergence , 2012, Journal of Mathematical Imaging and Vision.

[30]  Petter Holme,et al.  Structure and time evolution of an Internet dating community , 2002, Soc. Networks.

[31]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  G. Caldarelli,et al.  Systemic risk in financial networks , 2013 .

[33]  Ernesto Estrada,et al.  Statistical-mechanical approach to subgraph centrality in complex networks , 2007, 0905.4098.

[34]  A. Barabasi,et al.  Quantifying social group evolution , 2007, Nature.

[35]  R. Mantegna.  Hierarchical structure in financial markets , 1998, cond-mat/9802256.

[36]  Paul Erdös,et al.  On random graphs, I , 1959 .

[37]  C. Granger.  Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[38]  Jure Leskovec,et al.  Signed networks in social media , 2010, CHI.

[39]  J. Delvenne,et al.  Centrality measures and thermodynamic formalism for complex networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[40]  Francisco Escolano,et al.  Heat diffusion: thermodynamic depth complexity of networks. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[41]  S. Vetrivel,et al.  APPLICATIONS OF GRAPH THEORY IN COMPUTER SCIENCE AN OVERVIEW , 2010 .

[42]  Donald C. Mikulecky,et al.  Network Thermodynamics and Complexity: A Transition to Relational Systems Theory , 2001, Comput. Chem..

[43]  D. Garlaschelli,et al.  Emergence of Complexity in Financial Networks , 2004 .

[44]  Geoffrey Scott,et al.  The coefficients of the Ihara zeta function , 2008 .

[45]  Edwin R. Hancock,et al.  Spectral embedding of graphs , 2003, Pattern Recognit..

[46]  N. Rashevsky.  Life, information theory, and topology , 1955 .

[47]  Ian T. Foster,et al.  Mapping the Gnutella Network: Properties of Large-Scale Peer-to-Peer Systems and Implications for System Design , 2002, ArXiv.

[48]  Richard C. Wilson,et al.  Approximate von Neumann entropy for directed graphs. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[49]  Francisco A. Rodrigues,et al.  Collective behavior in financial markets , 2011 .

[50]  Kaspar Riesen,et al.  IAM Graph Database Repository for Graph Based Pattern Recognition and Machine Learning , 2008, SSPR/SPR.

[51]  Charles H. Bennett,et al.  On the nature and origin of complexity in discrete, homogeneous, locally-interacting systems , 1986 .

[52]  Matthias Dehmer,et al.  Advances in network complexity , 2013 .

[53]  Serkan Yuksel,et al.  Analysis of cross-correlations between financial markets after the 2008 crisis , 2013 .

[54]  Peter Grindrod,et al.  Evolving graphs: dynamical models, inverse problems and propagation , 2010, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[55]  Dmitri Krioukov,et al.  Entropy distribution and condensation in random networks with a given degree distribution. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[56]  Edwin R. Hancock,et al.  Fast depth-based subgraph kernels for unattributed graphs , 2016, Pattern Recognit..

[57]  E. Salmon.  Gene Expression During the Life Cycle of Drosophila melanogaster , 2002 .

[58]  Jon M. Kleinberg,et al.  Overview of the 2003 KDD Cup , 2003, SKDD.

[59]  Maarten van Steen,et al.  Graph Theory and Complex Networks: An Introduction , 2010 .

[60]  Edwin R. Hancock,et al.  Pattern Vectors from Algebraic Graph Theory , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  M E J Newman.  Assortative mixing in networks. , 2002, Physical review letters.

[62]  Ernesto Estrada.  Quantifying network heterogeneity. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[63]  C. Coulson,et al.  Hückel theory for organic chemists , 1978 .

[64]  V. Latora,et al.  Complex networks: Structure and dynamics , 2006 .

[65]  A. Shiryayev.  On Tables of Random Numbers , 1993 .

[66]  K. Kaski,et al.  Clustering and information in correlation based financial networks , 2003, cond-mat/0312682.

[67]  Edwin R. Hancock,et al.  Quantum walks, Ihara zeta functions and cospectrality in regular graphs , 2011, Quantum Inf. Process..

[68]  Edwin R. Hancock,et al.  Graph Characterization via Ihara Coefficients , 2011, IEEE Transactions on Neural Networks.

[69]  Albert,et al.  Topology of evolving networks: local events and universality , 2000, Physical review letters.

[70]  N. Biggs.  Algebraic Graph Theory: The multiplicative expansion , 1974 .

[71]  Robert Jenssen,et al.  Kernel Entropy Component Analysis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  Shaun M. Fallat,et al.  On the normalized Laplacian energy and general Randić index R-1 of graphs , 2010 .

[73]  Thomas Wilhelm,et al.  What is a complex graph , 2008 .

[74]  Xintian Zhuang,et al.  A network analysis of the Chinese stock market , 2009 .

[75]  Edwin R. Hancock,et al.  Heat Flow-Thermodynamic Depth Complexity in Directed Networks , 2012, SSPR/SPR.

[76]  A. Barabasi,et al.  Bose-Einstein condensation in complex networks. , 2000, Physical review letters.

[77]  Matthias Dehmer,et al.  Information processing in complex networks: Graph entropy and information functionals , 2008, Appl. Math. Comput..

[78]  Fabrizio Lillo,et al.  Cluster analysis for portfolio optimization , 2005, physics/0507006.

[79]  Michele Tumminello,et al.  Emergence of statistically validated financial intraday lead-lag relationships , 2014, 1401.0462.

[80]  Edwin R. Hancock,et al.  Graph characterizations from von Neumann entropy , 2012, Pattern Recognit. Lett..

[81]  Mark E. J. Newman,et al.  Structural inference for uncertain networks , 2015, Physical review. E.

[82]  Matthias Dehmer,et al.  Uniquely Discriminating Molecular Structures Using Novel Eigenvalue-Based Descriptors , 2012 .

[83]  Ernesto Estrada,et al.  The Structure of Complex Networks: Theory and Applications , 2011 .

[84]  Ginestra Bianconi.  Quantum statistics in complex networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[85]  Apostolos Katsaris,et al.  Speculative Bubbles in the S&P 500: Was the Tech Bubble Confined to the Tech Sector‘ , 2005 .

[86]  S. Fortunato,et al.  Statistical physics of social dynamics , 2007, 0710.3256.

[87]  Rosario N. Mantegna,et al.  Book Review: An Introduction to Econophysics, Correlations, and Complexity in Finance, N. Rosario, H. Mantegna, and H. E. Stanley, Cambridge University Press, Cambridge, 2000. , 2000 .

[88]  K Kaski,et al.  Time-dependent cross-correlations between different stock returns: a directed network of influence. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[89]  Leo Katz,et al.  A new status index derived from sociometric analysis , 1953 .

[90]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[91]  Ravi Kumar,et al.  Structure and evolution of online social networks , 2006, KDD '06.

[92]  Edwin R. Hancock,et al.  Graph characteristics from the heat kernel trace , 2009, Pattern Recognit..

[93]  F. Chung.  Laplacians and the Cheeger Inequality for Directed Graphs , 2005 .

[94]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[95]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[96]  Darryl Jenkins,et al.  Handbook of Airline Economics , 1995 .

[97]  Jianfeng Feng,et al.  Granger causality vs. dynamic Bayesian network inference: a comparative study , 2009, BMC Bioinformatics.

[98]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[99]  Maja Cevnik.  Applications of graph theory in communication networks , 2015 .

[100]  Lucas Antiqueira,et al.  Analyzing and modeling real-world phenomena with complex networks: a survey of applications , 2007, 0711.3199.

[101]  Lukasz Kaiser,et al.  Entanglement and the complexity of directed graphs , 2012, Theor. Comput. Sci..

[102]  Leonard M. Freeman,et al.  A set of measures of centrality based upon betweenness , 1977 .

[103]  Desmond J. Higham,et al.  Network Science - Complexity in Nature and Technology , 2010, Network Science.

[104]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[105]  Abbe Mowshowitz,et al.  Entropy and the complexity of graphs , 1967 .

[106]  G. Caldarelli,et al.  Networks of equities in financial markets , 2004 .

[107]  M Tumminello,et al.  A tool for filtering information in complex systems. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[108]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[109]  R. N. Mantegna,et al.  Identification of clusters of companies in stock indices via Potts super-paramagnetic transitions , 2000 .

[110]  Jacob G Foster,et al.  Edge direction and the structure of networks , 2009, Proceedings of the National Academy of Sciences.

[111]  Le Song,et al.  KELLER: estimating time-varying interactions between genes , 2009, Bioinform..

[112]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[113]  Geng Li,et al.  Effective graph classification based on topological and label attributes , 2012, Stat. Anal. Data Min..

[114]  Shunsuke Ihara,et al.  Information theory - for continuous systems , 1993 .

[115]  Kaspar Riesen,et al.  Graph Embedding in Vector Spaces by Means of Prototype Selection , 2007, GbRPR.

[116]  Edwin R. Hancock,et al.  A polynomial characterization of hypergraphs using the Ihara zeta function , 2011, Pattern Recognit..

[117]  Edwin R. Hancock,et al.  Structural Graph Matching Using the EM Algorithm and Singular Value Decomposition , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[118]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[119]  Ginestra Bianconi,et al.  Entropy measures for networks: toward an information theory of complex topologies. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[120]  Hisashi Kashima,et al.  Marginalized Kernels Between Labeled Graphs , 2003, ICML.

[121]  Abraham Kandel,et al.  Applied Graph Theory in Computer Vision and Pattern Recognition , 2007 .

[122]  Giuliano Armano,et al.  Quantum–classical transitions in complex networks , 2012, 1205.1771.