Structural Information and Dynamical Complexity of Networks

In 1953, Shannon proposed the question of quantification of structural information to analyze communication systems. The question has become one of the longest great challenges in information science and computer science. Here, we propose the first metric for structural information. Given a graph G , we define the K-dimensional structural information of G (or structure entropy of G), denoted by HK(G) , to be the minimum overall number of bits required to determine the K-dimensional code of the node that is accessible from random walk in G. The K-dimensional structural information provides the principle for completely detecting the natural or true structure, which consists of the rules, regulations, and orders of the graphs, for fully distinguishing the order from disorder in structured noisy data, and for analyzing communication systems, solving the Shannon's problem and opening up new directions. The K-dimensional structural information is also the first metric of dynamical complexity of networks, measuring the complexity of interactions, communications, operations, and even evolution of networks. The metric satisfies a number of fundamental properties, including additivity, locality, robustness, local and incremental computability, and so on. We establish the fundamental theorems of the one- and two-dimensional structural information of networks, including both lower and upper bounds of the metrics of classic data structures, general graphs, the networks of models, and the networks of natural evolution. We propose algorithms to approximate the K-dimensional structural information of graphs by finding the K-dimensional structure of the graphs that minimizes the K-dimensional structure entropy. We find that the K-dimensional structure entropy minimization is the principle for detecting the natural or true structures in real-world networks. Consequently, our structural information provides the foundation for knowledge discovering from noisy data. We establish a black hole principle by using the two-dimensional structure information of graphs. We propose the natural rank of locally listing algorithms by the structure entropy minimization principle, providing the basis for a next-generation search engine.

[1]  Angsheng Li,et al.  Three-Dimensional Gene Map of Cancer Cell Types: Structural Entropy Minimisation Principle for Defining Tumour Subtypes , 2016, Scientific Reports.

[2]  Albert-László Barabási,et al.  Hierarchical organization in complex networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  N. Trinajstic,et al.  Information theory, distance matrix, and molecular branching , 1977 .

[4]  Angsheng Li,et al.  Homophyly/Kinship Model: Naturally Evolving Networks , 2015, Scientific Reports.

[5]  Fan Chung Graham,et al.  Local Graph Partitioning using PageRank Vectors , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[6]  Frederick P. Brooks,et al.  Three great challenges for half-century-old computer science , 2003, JACM.

[7]  F. Chung,et al.  Complex Graphs and Networks , 2006 .

[8]  A. Vázquez.  Growing network with local rules: preferential attachment, clustering hierarchy, and degree correlations. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[9]  S. Severini,et al.  The Laplacian of a Graph as a Density Matrix: A Basic Combinatorial Approach to Separability of Mixed States , 2004, quant-ph/0406165.

[10]  Ash A. Alizadeh,et al.  Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling , 2000, Nature.

[11]  Allan Sly,et al.  Random graphs with a given degree sequence , 2010, 1005.1136.

[12]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[13]  H. Chernoff.  A Note on an Inequality Involving the Normal Distribution , 1981 .

[14]  Ginestra Bianconi,et al.  Entropy measures for networks: toward an information theory of complex topologies. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Pan Peng,et al.  Community Structures in Classical Network Models , 2011, Internet Math..

[16]  Jon M. Kleinberg,et al.  The small-world phenomenon: an algorithmic perspective , 2000, STOC '00.

[17]  Angsheng Li,et al.  A Theory of Network Security: Principles of Natural Selection and Combinatorics , 2016, Internet Math..

[18]  A. Bennett.  The Origin of Species by means of Natural Selection; or the Preservation of Favoured Races in the Struggle for Life , 1872, Nature.

[19]  B. Bollobás.  The evolution of random graphs , 1984 .

[20]  Abraham D. Flaxman,et al.  Expansion and Lack Thereof in Randomly Perturbed Graphs , 2007, Internet Math..

[21]  Martin Rosvall,et al.  Maps of random walks on complex networks reveal community structure , 2007, Proceedings of the National Academy of Sciences.

[22]  Avrim Blum,et al.  A Random-Surfer Web-Graph Model , 2006, ANALCO.

[23]  G. Bianconi,et al.  Shannon and von Neumann entropy of random networks with heterogeneous expected degree. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[24]  A. Mowshowitz,et al.  Entropy and the complexity of graphs. I. An index of the relative complexity of a graph. , 1968, The Bulletin of mathematical biophysics.

[25]  M E J Newman,et al.  Finding and evaluating community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[26]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[27]  G. Bianconi.  Entropy of network ensembles. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  Eli Upfal,et al.  Stochastic models for the Web graph , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[29]  Béla Bollobás,et al.  The Diameter of a Scale-Free Random Graph , 2004, Comb..

[30]  Wojciech Szpankowski,et al.  Compression of graphical structures , 2009, 2009 IEEE International Symposium on Information Theory.

[31]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[32]  N. Rashevsky.  Life, information theory, and topology , 1955 .

[33]  E. Trucco.  A note on the information content of graphs , 1956 .

[34]  Boris G. Pittel,et al.  Note on the Heights of Random Recursive Trees and Random m-ary Search Trees , 1994, Random Struct. Algorithms.

[35]  Matthias Dehmer,et al.  Information processing in complex networks: Graph entropy and information functionals , 2008, Appl. Math. Comput..

[36]  Ravi Kumar,et al.  Trawling the Web for Emerging Cyber-Communities , 1999, Comput. Networks.

[37]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[38]  Angsheng Li,et al.  Discovering natural communities in networks , 2015 .

[39]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[40]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[41]  Wei Zhang,et al.  Strategies for network security , 2014, Science China Information Sciences.

[42]  M. Newman,et al.  Finding community structure in very large networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  C. Raychaudhury,et al.  Discrimination of isomeric structures using information theoretic topological indices , 1984 .

[44]  Matthias Dehmer,et al.  A history of graph entropy measures , 2011, Inf. Sci..

[45]  Bruce A. Reed,et al.  A Critical Point for Random Graphs with a Given Degree Sequence , 1995, Random Struct. Algorithms.

[46]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[47]  Amin Saberi,et al.  On certain connectivity properties of the Internet topology , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[48]  P. Pin,et al.  Assessing the relevance of node features for network structure , 2008, Proceedings of the National Academy of Sciences.

[49]  Matthias Dehmer,et al.  Towards Network Complexity , 2009, Complex.

[50]  Maykel Cruz-Monteagudo,et al.  Markov Entropy Centrality: Chemical, Biological, Crime, and Legislative Networks , 2011, Towards an Information Theory of Complex Networks.

[51]  Claude E. Shannon,et al.  The lattice theory of information , 1953, Trans. IRE Prof. Group Inf. Theory.