Does Algorithmic Probability Solve the Problem of Induction

We will begin with a definition of Algorithmic Probability (ALP), and discuss some of its properties. From these remarks it will become clear that it is extremely effective for computing probabilities of future events — the best technique we have. As such, it gives us an ideal theoretical solution to the problem of inductive inference. I say “theoretical” because any device as accurate as ALP must necessarily be incomputable. For practical induction we use a set of approximations of increasing power to approach ALP. This set is called Resource Bounded Probability (RBP), and it constitutes a general solution to the problem of practical induction. Some of its properties are quite different from those of ALP. The rest of the paper will discuss philosophical and practical implications of the properties of ALP and RBP. It should be noted that the only argument that need be considered for the use of these techniques in induction is their effectiveness in getting good probability values for future events. Whether their properties are in accord with our intuitions about induction is a peripheral issue. The main question is “Do they work?” As we shall see, they do work.

[1]  N. Goodman,et al.  The Structure of Appearance. , 1953 .

[2]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[3]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[4]  Per Martin-Löf,et al.  The Definition of Random Sequences , 1966, Inf. Control..

[5]  Gregory J. Chaitin,et al.  On the Length of Programs for Computing Finite Binary Sequences , 1966, JACM.

[6]  A. Kolmogorov Three approaches to the quantitative definition of information , 1968 .

[7]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[8]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[9]  David G. Willis,et al.  Computational Complexity and Probability Constructions , 1970, JACM.

[10]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[11]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[12]  Mark S. Boddy,et al.  An Analysis of Time-Dependent Planning , 1988, AAAI.

[13]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 1997, Texts in Computer Science.

[14]  Erik Ordentlich,et al.  Universal portfolios with side information , 1996, IEEE Trans. Inf. Theory.

[15]  Kathryn B. Laskey Model uncertainty: theory and practical implications , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[16]  Jürgen Schmidhuber,et al.  Low-Complexity Art , 2017 .

[17]  R. Solomonoff A PRELIMINARY REPORT ON A GENERAL THEORY OF INDUCTIVE INFERENCE , 2001 .

[18]  J. Stochastic Complexity in Learning , .