General linear relations between different types of predictive complexity

In this paper we introduce a general method of establishing tight linear inequalities between different types of predictive complexity. Predictive complexity is a generalization of Kolmogorov complexity and it bounds the ability of an algorithm to predict elements of a sequence. Our method relies upon probabilistic considerations and allows us to describe explicitly the sets of coefficients which correspond to true inequalities. We apply this method to two particular types of predictive complexity, namely, logarithmic complexity, which coincides with a variant of Kolmogorov complexity, and square-loss complexity, which is interesting for applications.

[1]  David Haussler,et al.  Tight worst-case loss bounds for predicting with expert advice , 1994, EuroCOLT.

[2]  Vladimir Vovk,et al.  Universal portfolio selection , 1998, COLT' 98.

[3]  Jorma Rissanen,et al.  Hypothesis Selection and Testing by the MDL Principle , 1999, Comput. J..

[4]  Alexander Gammerman,et al.  Complexity Approximation Principle , 1999, Comput. J..

[5]  Philip D. Plowright,et al.  Convexity , 2019, Optimization for Chemical and Biochemical Engineering.

[6]  David Haussler,et al.  How to use expert advice , 1993, STOC.

[7]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[8]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[9]  David L. Dowe,et al.  Minimum Message Length and Kolmogorov Complexity , 1999, Comput. J..

[10]  Alfredo De Santis,et al.  Learning Probabilistic Prediction Functions (Extended Abstract) , 1988, FOCS 1988.

[11]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[12]  Vladimir Vovk,et al.  A game of prediction with expert advice , 1995, COLT '95.

[13]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[14]  Vladimir Vovk Probability theory for the Brier game , 2001, Theor. Comput. Sci..

[15]  R. Courant Differential and Integral Calculus , 1935 .