Complexity Monotone in Conditions and Future Prediction Errors

We bound the future loss when predicting any (computably) stochastic sequence online. Solomono finitely bounded the total devi- ation of his universal predictor M from the true distribution µ by the algorithmic complexity of µ. Here we assume we are at a time t > 1 and already observed x=x1...xt. We bound the future prediction per- formance on xt+1xt+2... by a new variant of algorithmic complexity of µ given x, plus the complexity of the randomness deficiency of x. The new complexity is monotone in its condition in the sense that this complexity can only decrease if the condition is prolonged. We also briefly discuss potential generalizations to Bayesian model classes and to classification problems.

[1]  Marcus Hutter.  Convergence and Loss Bounds for Bayesian Sequence Prediction , 2003, IEEE Trans. Inf. Theory.

[2]  Paul M. B. Vitányi,et al.  Clustering by compression , 2003, IEEE Transactions on Information Theory.

[3]  Marcus Hutter.  Convergence and Error Bounds for Universal Prediction of Nonbinary Sequences , 2001, ECML.

[4]  Jürgen Schmidhuber,et al.  Algorithmic Theories of Everything , 2000, ArXiv.

[5]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[6]  Marcus Hutter.  Sequence Prediction Based on Monotone Complexity , 2003, COLT.

[7]  Marcus Hutter.  New Error Bounds for Solomonoff Prediction , 2001, J. Comput. Syst. Sci..

[8]  Marcus Hutter,et al.  Convergence of Discrete MDL for Sequential Prediction , 2004, COLT.

[9]  Jürgen Schmidhuber,et al.  Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit , 2002, Int. J. Found. Comput. Sci..

[10]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[11]  Marcus Hutter.  Optimality of universal Bayesian prediction for general loss and alphabet , 2003 .

[12]  Marcus Hutter.  Sequential Predictions based on Algorithmic Complexity , 2006, J. Comput. Syst. Sci..

[13]  Marcus Hutter.  General Loss Bounds for Universal Sequence Prediction , 2001, ICML.

[14]  Jürgen Schmidhuber,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002, COLT.

[15]  Alexander Shen,et al.  Relations between varieties of kolmogorov complexities , 1996, Mathematical systems theory.

[16]  Marcus Hutter,et al.  Universal Artificial Intellegence - Sequential Decisions Based on Algorithmic Probability , 2005, Texts in Theoretical Computer Science. An EATCS Series.

[17]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[18]  Marcus Hutter,et al.  Universal Convergence of Semimeasures on Individual Random Sequences , 2004, ALT.

[19]  L. Levin,et al.  THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS , 1970 .

[20]  Ofi rNw8x'pyzm,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002 .