A UNIFYING FRAMEWORK FOR COMPLEXITY MEASURES OF FINITE SYSTEMS

We develop a unifying approach for complexity measures, based on the principle that complexity requires interactions at different scales of description. Complex systems are more than the sum of their parts of any size, and not just more than the sum of their elements. We therefore analyze the decomposition of a system in terms of an interaction hierarchy. In mathematical terms, we present a theory of complexity measures for finite random fields using the geometric framework of hierarchies of exponential families. Within our framework, previously proposed complexity measures find their natural place and gain a new interpretation.

[1]  Salvatore D. Morgera,et al.  Information theoretic covariance complexity and its relation to pattern recognition , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  P. Grassberger.  Toward a quantitative theory of self-generated complexity , 1986 .

[3]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  G. Edelman,et al.  A measure for brain complexity: relating functional segregation and integration in the nervous system. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[6]  M. Studený,et al.  The Multiinformation Function as a Tool for Measuring Stochastic Dependence , 1998, Learning in Graphical Models.

[7]  Shun-ichi Amari,et al.  Information geometry on hierarchy of probability distributions , 2001, IEEE Trans. Inf. Theory.

[8]  A. U.S.,et al.  Predictability , Complexity , and Learning , 2002 .

[9]  N. Ay.  AN INFORMATION-GEOMETRIC APPROACH TO A THEORY OF PRAGMATIC STRUCTURING , 2002 .

[10]  N. Ay,et al.  On maximization of the information divergence from an exponential family , 2003 .

[11]  Thomas Wennekers,et al.  Dynamical properties of strongly interacting Markov chains , 2003, Neural Networks.

[12]  J. Crutchfield,et al.  Regularities unseen, randomness observed: levels of entropy convergence. , 2001, Chaos.

[13]  James P. Crutchfield,et al.  Synchronizing to Periodicity: the Transient Information and Synchronization Time of Periodic Sequences , 2002, Adv. Complex Syst..

[14]  C.J.H. Mann,et al.  Probabilistic Conditional Independence Structures , 2005 .

[15]  Nihat Ay,et al.  Maximizing Multi-Information , 2007 .