Algorithmic Probability|Theory and Applications

We flrst deflne Algorithmic Probability, an extremely powerful method of inductive inference. We discuss its completeness, incomputability, diversity and subjectivity and show that its incomputability in no way inhibits its use for practical prediction. Applications to Bernoulli sequence prediction and grammar discovery are described. We conclude with a note on its employment in a very strong AI system for very general problem solving.

[1]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[2]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[3]  Andreas Stolcke,et al.  Inducing Probabilistic Grammars by Bayesian Model Merging , 1994, ICGI.

[4]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[5]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[6]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[7]  Hussein A. Abbass,et al.  Grammar model-based program evolution , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[8]  R. Solomonoff.  Progress In Incremental Machine Learning , 2003 .

[9]  R. Solomonoff.  A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY , 1989 .

[10]  Ray J. Solomonofi,et al.  A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY , 1989 .

[11]  Andreas Stolcke,et al.  Precise N-Gram Probabilities From Stochastic Context-Free Grammars , 1994, ACL.

[12]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[13]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[14]  Noam Chomsky.  Learning Context-free Grammars , .

[15]  R. Solomonoff.  A PRELIMINARY REPORT ON A GENERAL THEORY OF INDUCTIVE INFERENCE , 2001 .

[16]  Ray J. Solomonoff,et al.  The Discovery of Algorithmic Probability , 1997, J. Comput. Syst. Sci..

[17]  Colin de la Higuera.  Grammatical Inference: Learning context-free grammars , 2010 .

[18]  Ray J. Solomonoff,et al.  Three Kinds of Probabilistic Induction: Universal Distributions and Convergence Theorems , 2008, Comput. J..

[19]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .