Generalised entropies and asymptotic complexities of languages

The paper explores connections between asymptotic complexity and generalised entropy. Asymptotic complexity of a language (a language is a set of finite or infinite strings) is a way of formalising the complexity of predicting the next element in a sequence: it is the loss per element of a strategy asymptotically optimal for that language. Generalised entropy extends Shannon entropy to arbitrary loss functions; it is the optimal expected loss given a distribution on possible outcomes. It turns out that the set of tuples of asymptotic complexities of a language w.r.t. different loss functions can be described by means of the generalised entropies corresponding to the loss functions.

[1]  Vladimir Vovk,et al.  Universal portfolio selection , 1998, COLT' 98.

[2]  Vladimir Vovk,et al.  Loss functions, complexities, and the Legendre transformation , 2004, Theor. Comput. Sci..

[3]  Yuri Kalnishkan.  General linear relations between different types of predictive complexity , 2002, Theor. Comput. Sci..

[4]  John M. Hitchcock.  Fractal dimension and logarithmic loss unpredictability , 2003, Theor. Comput. Sci..

[5]  Yuri Kalnishkan,et al.  Genral Linear Relations among Different Types of Predictive Complexity , 1999, ALT.

[6]  Vladimir Vovk,et al.  Generalised Entropy and Asymptotic Complexities of Languages , 2007, COLT.

[7]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[8]  Daniil Ryabko.  On the Relation between Realizable and Nonrealizable Cases of the Sequence Prediction Problem , 2011, J. Mach. Learn. Res..

[9]  B. Gelbaum,et al.  Counterexamples in Analysis , 1967 .

[10]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[11]  Yuri Kalnishkan,et al.  The weak aggregating algorithm and weak mixability , 2008, J. Comput. Syst. Sci..

[12]  Klaus Weihrauch,et al.  Computable Analysis: An Introduction , 2014, Texts in Theoretical Computer Science. An EATCS Series.

[13]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[14]  Lance Fortnow,et al.  Prediction and dimension , 2005, J. Comput. Syst. Sci..

[15]  A. Dawid,et al.  Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory , 2004, math/0410076.

[16]  Vladimir Vovk,et al.  How many strings are easy to predict? , 2005, Inf. Comput..

[17]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[18]  Dr. Marcus Hutter,et al.  Universal artificial intelligence , 2004 .

[19]  Ker-I Ko,et al.  Complexity Theory of Real Functions , 1991, Progress in Theoretical Computer Science.