Complexity Approximation Principle

We propose a new inductive principle, which we call the complexity approximation principle (CAP). This principle is a natural generalization of Rissanen’s minimum description length (MDL) principle and Wallace’s minimum message length (MML) principle and is based on the notion of predictive complexity, a recent generalization of Kolmogorov complexity. Like the MDL principle, CAP can be regarded as an implementation of Occam’s razor.

[1]  Vladimir Vovk,et al.  Probability theory for the Brier game , 1997, Theor. Comput. Sci..

[2]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[3]  Ming Li,et al.  Kolmogorov Complexity and its Applications , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[4]  David Haussler,et al.  Tight worst-case loss bounds for predicting with expert advice , 1994, EuroCOLT.

[5]  J. Rissanen.  A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[6]  Erik Ordentlich,et al.  Universal portfolios with side information , 1996, IEEE Trans. Inf. Theory.

[7]  Vladimir V. V'yugin,et al.  Algorithmic Complexity and Stochastic Properties of Finite Binary Sequences , 1999, Comput. J..

[8]  J. Lewins.  Contribution to the Discussion , 1989 .

[9]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[10]  Jorma Rissanen,et al.  Hypothesis Selection and Testing by the MDL Principle , 1999, Comput. J..

[11]  C. Q. Lee,et al.  The Computer Journal , 1958, Nature.

[12]  Vladimir Vovk,et al.  A game of prediction with expert advice , 1995, COLT '95.

[13]  Alexander Gammerman,et al.  Kolmogorov Complexity: Sources, Theory and Applications , 1999, Comput. J..

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  Vladimir Vovk,et al.  Universal portfolio selection , 1998, COLT' 98.

[16]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[17]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[18]  C. S. Wallace,et al.  Estimation and Inference by Compact Coding , 1987 .