Two decades of applied Kolmogorov complexity: in memoriam Andrei Nikolaevich Kolmogorov 1903-87

The authors provide an introduction to the main ideas of Kolmogorov complexity and survey the wealth of useful applications of this notion. It is based on a theory of information content of strings, intuitively, that the amount of information in a finite string is the size (i.e. number of bits) of the smallest program that, started with a blank memory, computes the string and then terminates. The following are treated: (1) application of the fact that some strings are compressible; this includes a strong version of Godel's incompleteness theorem; (2) lower-bound arguments that rest on application of the fact that certain strings cannot be compressed at all; applications range from Turing machines to electronic chips; (3) probability theory and a priori probability; applications range from foundational issues to the theory of learning and inductive inference in artificial intelligence. (4) Resource-bounded Kolmogorov complexity; applications range from NP-completeness and the P versus NP question to cryptography.<<ETX>>

[1]  Osamu Watanabe,et al.  On Exponential Lowness , 1986, ICALP.

[2]  Osamu Watanabe,et al.  Kolmogorov complexity and degrees of tally sets , 1988, [1988] Proceedings. Structure in Complexity Theory Third Annual Conference.

[3]  Juris Hartmanis,et al.  On Sparse Oracles Separating Feasible Complexity Classes , 1988, Inf. Process. Lett..

[4]  Donald W. Loveland,et al.  On minimal-program complexity measures , 1969, STOC.

[5]  Oscar H. Ibarra,et al.  Multi-Tape and Multi-Head Pushdown Automata , 1968, Inf. Control..

[6]  Ming Li.  Lower bounds in Computational Complexity , 1985 .

[7]  Ming Li,et al.  String-Matching Cannot be Done by a Two-Head One-Way Deterministic Finite Automation , 1986, Inf. Process. Lett..

[8]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[9]  A. Church.  On the concept of a random sequence , 1940 .

[10]  Janos Simon,et al.  Lower bounds on the time of probabilistic on-line simulations , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[11]  Paul M. B. Vitányi,et al.  Big omega versus the wild functions , 1985, SIGA.

[12]  P. Martin-Löf.  Complexity oscillations in infinite binary sequences , 1971 .

[13]  Gregory J. Chaitin,et al.  On the Length of Programs for Computing Finite Binary Sequences , 1966, JACM.

[14]  W. Maass.  Combinatorial lower bound arguments for deterministic and nondeterministic Turing machines , 1985 .

[15]  Claus-Peter Schnorr,et al.  Process complexity and effective random tests , 1973 .

[16]  José L. Balcázar,et al.  On Generalized Kolmogorov Complexity , 1986, STACS.

[17]  G. Chaitin.  Randomness and Mathematical Proof , 1975 .

[18]  Martin Dietzfelbinger,et al.  Lower bounds on computation time for various models in computational complexity theory , 1987 .

[19]  Michael Sipser,et al.  A complexity theoretic approach to randomness , 1983, STOC.

[20]  Gregory J. Chaitin,et al.  On the Length of Programs for Computing Finite Binary Sequences: statistical considerations , 1969, JACM.

[21]  Vijay V. Vazirani,et al.  A natural encoding scheme proved probabilistic polynomial complete , 1982, 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982).

[22]  Per Martin-Löf,et al.  The Definition of Random Sequences , 1966, Inf. Control..

[23]  G. Chaitin.  Gödel's theorem and information , 1982 .

[24]  B. Gnedenko,et al.  Andrei Nikolaevich Kolmogorov (on his eightieth birthday) , 1983 .

[25]  守屋 悦朗,et al.  J.E.Hopcroft, J.D. Ullman 著, "Introduction to Automata Theory, Languages, and Computation", Addison-Wesley, A5変形版, X+418, \6,670, 1979 , 1980 .

[26]  Juris Hartmanis,et al.  On Isomorphisms and Density of NP and Other Complete Sets , 1977, SIAM J. Comput..

[27]  P S Aleksandrov.  A few words on A. N. Kolmogorov , 1983 .

[28]  Endre Szemerédi,et al.  Two tapes are better than one for off-line Turing machines , 1987, STOC '87.

[29]  I. HAL SUDBOROUGH.  One-Way Multihead Writing Finite Automata , 1976, Inf. Control..

[30]  Gary L. Peterson,et al.  Succinct representation random strings, and complexity classes , 1980, 21st Annual Symposium on Foundations of Computer Science (sfcs 1980).

[31]  David G. Willis,et al.  Computational Complexity and Probability Constructions , 1970, JACM.

[32]  Wolfgang J. Paul.  On Heads Versus Tapes , 1984, Theor. Comput. Sci..

[33]  Andrei N. Kolmogorov,et al.  Logical basis for information theory and probability theory , 1968, IEEE Trans. Inf. Theory.

[34]  Paul Beame,et al.  Limits on the power of concurrent-write parallel machines , 1986, STOC '86.

[35]  Gregory J. Chaitin,et al.  Information-Theoretic Limitations of Formal Systems , 1974, JACM.

[36]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[37]  Andrew Chi-Chih Yao,et al.  Theory and application of trapdoor functions , 1982, 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982).

[38]  Donald W. Loveland,et al.  A Variant of the Kolmogorov Concept of Complexity , 1969, Information and Control.

[39]  Ming Li,et al.  New lower bounds for parallel computation , 1986, STOC '86.

[40]  Ming Li,et al.  Tape versus Queue and Stacks: The Lower Bounds , 1988, Inf. Comput..

[41]  Donald E. Knuth,et al.  Big Omicron and big Omega and big Theta , 1976, SIGA.

[42]  Ivan Hal Sudborough,et al.  Computation by multi-head writing finite automata , 1971 .

[43]  C. Thomborson,et al.  Area-time complexity for VLSI , 1979, STOC.

[44]  Robert P. Daley.  On the Inference of Optimal Descriptions , 1977, Theor. Comput. Sci..

[45]  Gregory J. Chaitin,et al.  Algorithmic Information Theory , 1987, IBM J. Res. Dev..

[46]  Allan Borodin,et al.  A time-space tradeoff for sorting and related non-oblivious computations , 1979, SIGA.

[47]  Juris Hartmanis,et al.  Generalized Kolmogorov complexity and the structure of feasible computations , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[48]  Claude E. Shannon,et al.  The Mathematical Theory of Communication , 1950 .

[49]  Georg Schnitger,et al.  An Optimal Lower Bound for Turing Machines with One Work Tape and a Two- way Input Tape , 1986, Computational Complexity Conference.

[50]  Zvi Galil,et al.  Time-Space-Optimal String Matching , 1983, J. Comput. Syst. Sci..

[51]  Leonid A. Levin,et al.  Randomness Conservation Inequalities; Information and Independence in Mathematical Theories , 1984, Inf. Control..

[52]  R. Cuykendall.  Kolmogorov information and vlsi lower bounds , 1984 .

[53]  Thomas Tymoczko.  New Directions in the Philosophy of Mathematics , 1985 .

[54]  Allan Borodin,et al.  A time-space tradeoff for sorting on a general sequential model of computation , 1980, STOC '80.

[55]  Leslie G. Valiant,et al.  Universal schemes for parallel communication , 1981, STOC '81.

[56]  Richard J. Lipton,et al.  Lower bounds for VLSI , 1981, STOC '81.

[57]  Satoru Miyano,et al.  Remarks on Multihead Pushdown Automata and Multihead Stack Automata , 1983, J. Comput. Syst. Sci..

[58]  David Haussler,et al.  Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension , 1986, STOC '86.

[59]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[60]  Eric Allender,et al.  Some consequences of the existence of pseudorandom generators , 1987, J. Comput. Syst. Sci..

[61]  P. Laplace.  A Philosophical Essay On Probabilities , 1902 .

[62]  G. J. Chaltin,et al.  To a mathematical definition of 'life' , 1970, SIGA.

[63]  D. Champernowne.  The Construction of Decimals Normal in the Scale of Ten , 1933 .

[64]  C. Schnorr.  A Survey of the Theory of Random Sequences , 1977 .