Kolmogorov Complexity in perspective. Part I: Information Theory and Randomnes

We survey diverse approaches to the notion of information: from Shannon entropy to Kolmogorov complexity. Two of the main applications of Kolmogorov complexity are presented: randomness and classification. The survey is divided in two parts in the same volume. Part I is dedicated to information theory and the mathematical formalization of randomness based on Kolmogorov complexity. This last application goes back to the 1960s and 1970s with the work of Martin-Lof, Schnorr, Chaitin, Levin, and has gained new impetus in the last years.

[1]  Panu Raatikainen.  On Interpreting Chaitin's Incompleteness Theorem , 1998, J. Philos. Log..

[2]  Claus-Peter Schnorr,et al.  A unified approach to the definition of random sequences , 1971, Mathematical systems theory.

[3]  Liang Yu,et al.  On initial segment complexity and degrees of randomness , 2008 .

[4]  Gregory J. Chaitin,et al.  Information-Theoretic Limitations of Formal Systems , 1974, JACM.

[5]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[6]  W. Feller,et al.  An Introduction to Probability Theory and Its Applications, Vol. 1 , 1967 .

[7]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[8]  Joan Rand Moschovakis,et al.  An intuitionistic theory of lawlike, choice and lawless sequences , 1993 .

[9]  D. Huffman.  A Method for the Construction of Minimum-Redundancy Codes , 1952 .

[10]  V. Becher,et al.  From index sets to randomness in ∅n: random reals and possibly infinite computations part II , 2009, The Journal of Symbolic Logic.

[11]  Gregory J. Chaitin,et al.  A recent technical report , 1974, SIGA.

[12]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[13]  Robert I. Soare,et al.  Computability and Recursion , 1996, Bulletin of Symbolic Logic.

[14]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[15]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[16]  Santiago Figueira,et al.  Randomness and halting probabilities , 2006, Journal of Symbolic Logic.

[17]  Gregory J. Chaitin,et al.  On the Length of Programs for Computing Finite Binary Sequences: statistical considerations , 1969, JACM.

[18]  Marie Ferbus-Zanda,et al.  Kolmogorov complexity and set theoretical representations of integers , 2006, Math. Log. Q..

[19]  Annick Lesne,et al.  L'héritage de Kolmogorov en mathématiques , 2004 .

[20]  Erhard Tornier,et al.  Grundlagen der Wahrscheinlichkeitsrechnung , 1933 .

[21]  Andrei Voronkov,et al.  Current Trends in Theoretical Computer Science , 2001 .

[22]  Verónica Becher,et al.  Random reals à la Chaitin with or without prefix-freeness , 2007, Theor. Comput. Sci..

[23]  Marie Ferbus-Zanda,et al.  Is Randomness "Native" to Computer Science? , 2008, Bull. EATCS.

[24]  J. Heijenoort.  From Frege to Gödel: A Source Book in Mathematical Logic, 1879-1931 , 1967 .

[25]  Joan Rand Moschovakis,et al.  More about relatively lawless sequences , 1994, Journal of Symbolic Logic.

[26]  Wolfgang Merkle,et al.  Reconciling Data Compression and Kolmogorov Complexity , 2007, ICALP.

[27]  Donald Ervin Knuth,et al.  The Art of Computer Programming, 2nd Ed. (Addison-Wesley Series in Computer Science and Information , 1978 .

[28]  Claus-Peter Schnorr,et al.  The process complexity and effective random tests. , 1972, STOC.

[29]  Don Fallis.  The Source of Chaitin's Incorrectness , 1996 .

[30]  A. Shiryayev.  On Tables of Random Numbers , 1993 .

[31]  A. A..  Probability, Statistics and Truth , 1940, Nature.

[32]  S. S. Wilks,et al.  Probability, statistics and truth , 1939 .

[33]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[34]  Hector Zenil.  Randomness Through Computation: Some Answers, More Questions , 2011 .

[35]  A. N. Kolmogorov.  Combinatorial foundations of information theory and the calculus of probabilities , 1983 .

[36]  C. Schnorr.  Zufälligkeit und Wahrscheinlichkeit , 1971 .

[37]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[38]  Joan Rand Moschovakis,et al.  A Classical View of the Intuitionistic Continuum , 1996, Ann. Pure Appl. Log..

[39]  G. Chaitin.  Computational complexity and Gödel's incompleteness theorem , 1971, SIGA.

[40]  Cristian S. Calude,et al.  Is complexity a source of incompleteness? , 2004, Adv. Appl. Math..

[41]  Gregory J. Chaitin,et al.  On the Length of Programs for Computing Finite Binary Sequences , 1966, JACM.

[42]  Péter Gács,et al.  Exact Expressions for Some Randomness Tests , 1979, Math. Log. Q..

[43]  Marcus Hutter,et al.  Algorithmic Information Theory , 1977, IBM J. Res. Dev..

[44]  Wolfgang Merkle,et al.  A Simple Proof of Miller-Yu Theorem , 2008, Fundam. Informaticae.

[45]  André Nies,et al.  Program Size Complexity for Possibly Infinite Computations , 2005, Notre Dame J. Formal Log..

[46]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[47]  Per Martin-Löf,et al.  The Definition of Random Sequences , 1966, Inf. Control..

[48]  André Nies,et al.  Randomness, relativization and Turing degrees , 2005, J. Symb. Log..

[49]  Andrei N. Kolmogorov,et al.  On Tables of Random Numbers (Reprinted from "Sankhya: The Indian Journal of Statistics", Series A, Vol. 25 Part 4, 1963) , 1998, Theor. Comput. Sci..

[50]  Donald Ervin Knuth,et al.  The Art of Computer Programming, Volume II: Seminumerical Algorithms , 1970 .

[51]  B. Russell.  Mathematical Logic as Based on the Theory of Types , 1908 .

[52]  Guillaume Bonfante,et al.  On Abstract Computer Virology from a Recursion Theoretic Perspective , 2006, Journal in Computer Virology.

[53]  A. Kolmogoroff.  Grundbegriffe der Wahrscheinlichkeitsrechnung , 1933 .

[54]  Richard Von Mises,et al.  Probability, statistics and truth , 1939 .

[55]  R. Mises.  Grundlagen der Wahrscheinlichkeitsrechnung , 1919 .

[56]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[57]  P. Martin-Lof,et al.  Complexity Oscillations in Infinite Binary Sequences , 2004 .

[58]  A. N. Kolmogorov,et al.  Foundations of the theory of probability , 1960 .

[59]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[60]  Jean-Paul Delahaye,et al.  Complexités : Aux limites des mathématiques et de l'informatique , 2006 .

[61]  Claus-Peter Schnorr,et al.  Process complexity and effective random tests , 1973 .

[62]  D. Lacombe,et al.  La théorie des fonctions récursives et ses applications. (Exposé d'information générale) , 1960 .

[63]  Max L. Warshauer,et al.  Lecture Notes in Mathematics , 2001 .

[64]  Verónica Becher,et al.  Random reals and possibly infinite computations Part I: Randomness in ∅′ , 2005, Journal of Symbolic Logic.