Intuitively, the amount of information in a string is the size of the shortest program that outputs the string. The first billion digits of π for example, contain very little information, since they can be calculated by a C program of a few lines only. Although information content seems to be highly dependent on choice of programming language, the notion is actually invariant up to an additive constant. The theory of program size complexity, which has become known as Kolmogorov complexity after one of its founding fathers, has found fruitful application in many fields such as combinatorics, algorithm analysis, machine learning, machine models, and logic. In this paper we propose a concrete definition of Kolomogorov complexity that is (arguably) as simple as possible, by defining a machine model based on the elegantly minimal Combinatory Logic, and exhibiting a universal machine.
[1]
Maribel Fernández.
The Lambda Calculus
,
2009
.
[2]
Ming Li,et al.
An Introduction to Kolmogorov Complexity and Its Applications
,
2019,
Texts in Computer Science.
[3]
Gregory J. Chaitin.
An Invitation to Algorithmic Information Theory
,
1996,
DMTCS.
[4]
H. Stowell.
The emperor's new mind
R. Penrose, Oxford University Press, New York (1989) 466 pp. $24.95
,
1990,
Neuroscience.
[5]
Henk Barendregt,et al.
The Lambda Calculus: Its Syntax and Semantics
,
1985
.
[6]
Paul M. B. Vitányi,et al.
An Introduction to Kolmogorov Complexity and Its Applications
,
1993,
Graduate Texts in Computer Science.