Towards understanding the behavior of physical systems using information theory

One of the goals of complex network analysis is to identify the most influential nodes, i.e., the nodes that dictate the dynamics of other nodes. In the case of autonomous systems or transportation networks, highly connected hubs play a preeminent role in diffusing the flow of information and viruses; in contrast, in language evolution most linguistic norms come from the peripheral nodes who have only few contacts. Clearly a topological analysis of the interactions alone is not sufficient to identify the nodes that drive the state of the network. Here we show how information theory can be used to quantify how the dynamics of individual nodes propagate through a system. We interpret the state of a node as a storage of information about the state of other nodes, which is quantified in terms of Shannon information. This information is transferred through interactions and lost due to noise, and we calculate how far it can travel through a network. We apply this concept to a model of opinion formation in a complex social network to calculate the impact of each node by measuring how long its opinion is remembered by the network. Counter-intuitively we find that the dynamics of opinions are not determined by the hubs or peripheral nodes, but rather by nodes with an intermediate connectivity.

[1]  T. L. Duncan,et al.  Information Loss as a Foundational Principle for the Second Law of Thermodynamics , 2007 .

[2]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[3]  Lada A. Adamic,et al.  Comment to "Emergence of Scaling in Random Networks" (cond-mat/9910332) , 2000, cond-mat/0001459.

[4]  J. Coleman,et al.  Medical Innovation: A Diffusion Study. , 1967 .

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Werner Ebeling,et al.  Word frequency and entropy of symbolic sequences: a dynamical perspective , 1992 .

[7]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[8]  R. Landauer.  The physical nature of information , 1996 .

[9]  James P Crutchfield,et al.  Time's barbed arrow: irreversibility, crypticity, and stored information. , 2009, Physical review letters.

[10]  Marco De Spirito,et al.  Modifications in solvent clusters embedded along the fibers of a cellulose polymer network cause paper degradation. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[11]  F. Bass.  A new product growth model for consumer durables , 1976 .

[12]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[13]  D. Stauffer,et al.  Ferromagnetic phase transition in Barabási–Albert networks , 2001, cond-mat/0112312.

[14]  Bahram Mashhoon,et al.  Gravitational effects of rotating masses , 1985 .

[15]  Albert-László Barabási,et al.  Controllability of complex networks , 2011, Nature.

[16]  Bahram Mashhoon,et al.  Relativistic Effects in the Motion of the Moon , 2000, gr-qc/0003077.

[17]  J. Crutchfield,et al.  Statistical complexity of simple one-dimensional spin systems , 1997, cond-mat/9702191.

[18]  R. Bagozzi,et al.  Intentional social action in virtual communities , 2002 .

[19]  B. Ryan.  The diffusion of hybrid seed corn in two Iowa communities , 1943 .

[20]  Shun-Chen Niu,et al.  A Stochastic Formulation of the Bass Model of New-Product Diffusion , 2002 .

[21]  S. Lloyd.  Programming the Universe: A Quantum Computer Scientist Takes on the Cosmos , 2006 .

[22]  S. Fortunato,et al.  Statistical physics of social dynamics , 2007, 0710.3256.

[23]  R. Bagozzi,et al.  A Social Influence Model of Consumer Participation in Network- and Small-Group-Based Virtual Communities , 2004 .

[24]  M. Macy,et al.  Complex Contagions and the Weakness of Long Ties1 , 2007, American Journal of Sociology.

[25]  S. Galam,et al.  Towards a theory of collective phenomena: Consensus and attitude changes in groups , 1991 .

[26]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[27]  S. Redner,et al.  Voter models on heterogeneous networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  J. Crutchfield,et al.  Regularities unseen, randomness observed: levels of entropy convergence. , 2001, Chaos.

[29]  Jure Leskovec,et al.  The dynamics of viral marketing , 2005, EC '06.

[30]  Karoline Wiesner,et al.  Nature computes: information processing in quantum dynamical systems. , 2010, Chaos.

[31]  S. Lloyd.  Ultimate physical limits to computation , 1999, Nature.

[32]  Albert-László Barabási,et al.  Error and attack tolerance of complex networks , 2000, Nature.

[33]  J. Coleman,et al.  Medical Innovation: A Diffusion Study. , 1967 .

[34]  Charles H. Bennett,et al.  Notes on Landauer's Principle, Reversible Computation, and Maxwell's Demon , 2002, physics/0210005.

[35]  Jens Timmer,et al.  Reconstructing gene-regulatory networks from time series, knock-out data, and prior knowledge , 2007, BMC Systems Biology.

[36]  R. Zecchina,et al.  Ferromagnetic ordering in graphs with arbitrary degree distribution , 2002, cond-mat/0203416.