Effective Complexity and Its Relation to Logical Depth

Effective complexity measures the information content of the regularities of an object. It has been introduced by Gell-Mann and Lloyd to avoid some of the disadvantages of Kolmogorov complexity. In this paper, we derive a precise definition of effective complexity in terms of algorithmic information theory. We analyze rigorously its basic properties such as effective simplicity of incompressible binary strings and existence of strings that have effective complexity close to their lengths. Since some of the results have appeared independently in the context of algorithmic statistics by Gács , we discuss the relation of effective complexity to the corresponding complexity measures, in particular to Kolmogorov minimal sufficient statistics. As our main new result we show a remarkable relation between effective complexity and Bennett's logical depth: If the effective complexity of a string x exceeds a certain explicit threshold then that string must have astronomically large depth; otherwise, the depth can be arbitrarily small.

[1]  Lance Fortnow,et al.  Sophistication Revisited , 2003, Theory of Computing Systems.

[2]  Nihat Ay,et al.  Effective Complexity of Stationary Process Realizations , 2011, Entropy.

[3]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  S. Lloyd,et al.  Measures of complexity: a nonexhaustive list , 2001 .

[6]  Jorma Rissanen,et al.  Information and Complexity in Statistical Modeling , 2006, ITW.

[7]  Seth Lloyd,et al.  Effective Complexity , 1995 .

[8]  J. Rissanen Stochastic Complexity in Statistical Inquiry Theory , 1989 .

[9]  Paul M. B. Vitányi,et al.  Meaningful Information , 2001, IEEE Transactions on Information Theory.

[10]  Gregory J. Chaitin,et al.  Exploring RANDOMNESS , 2001, Discrete Mathematics and Theoretical Computer Science.

[11]  T. Papaioannou Information, Measures of , 2006 .

[12]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[13]  Charles H. Bennett Logical depth and physical complexity , 1988 .

[14]  Rolf Herken,et al.  The Universal Turing Machine: A Half-Century Survey , 1992 .

[15]  Péter Gács,et al.  Algorithmic statistics , 2000, IEEE Trans. Inf. Theory.

[16]  Paul M. B. Vitányi,et al.  Randomness , 2001, ArXiv.

[17]  Seth Lloyd,et al.  Information measures, effective complexity, and total information , 1996, Complex..