Defensive Forecasting for Linear Protocols

We consider a general class of forecasting protocols, called “linear protocols”, and discuss several important special cases, including multi-class forecasting. Forecasting is formalized as a game between three players: Reality, whose role is to generate objects and their labels; Forecaster, whose goal is to predict the labels; and Skeptic, who tries to make money on any lack of agreement between Forecaster’s predictions and the actual labels. Our main mathematical result is that for any continuous strategy for Skeptic in a linear protocol there exists a strategy for Forecaster that does not allow Skeptic’s capital to grow. This result is a meta-theorem that allows one to transform any constructive law of probability in a linear protocol into a forecasting strategy whose predictions are guaranteed to satisfy this law. We apply this meta-theorem to a weak law of large numbers in inner product spaces to obtain a version of the K29 prediction algorithm for linear protocols and show that this version also satisfies the attractive properties of proper calibration and resolution under a suitable choice of its kernel parameter, with no assumptions about the way the data is generated.

[1]  M Morse,et al.  Singular Points of Vector Fields under General Boundary Conditions. , 1928, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Vladimir Vovk Non-asymptotic calibration and resolution , 2007, Theor. Comput. Sci..

[3]  William H. Press,et al.  Numerical recipes in C , 2002 .

[4]  G. Shafer,et al.  Probability and Finance: It's Only a Game! , 2001 .

[5]  J. V. Tucker,et al.  Computable and continuous partial homomorphisms on metric partial algebras , 2003, Bull. Symb. Log..

[6]  Y. Cho,et al.  Fixed Point Theory and Applications , 2000 .

[7]  A. Kolmogoroff Über das Gesetz des iterierten Logarithmus , 1929 .

[8]  William H. Press,et al.  Numerical Recipes in C, 2nd Edition , 1992 .

[9]  R. Schneider Convex Bodies: The Brunn–Minkowski Theory: Minkowski addition , 1993 .

[10]  M. Duflo,et al.  Sur la loi des grands nombres pour les martingales vectorielles et l'estimateur des moindres carrés d'un modèle de régression , 1990 .

[11]  Akimichi Takemura,et al.  Defensive Forecasting , 2005, AISTATS.

[12]  Vladimir Vovk Competitive on-line learning with a convex loss function , 2005, ArXiv.

[13]  Vladimir Vovk Defensive Prediction with Expert Advice , 2005, ALT.

[14]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[15]  Christine Thomas-Agnan,et al.  Computing a family of reproducing kernels for statistical applications , 1996, Numerical Algorithms.

[16]  N. Aronszajn Theory of Reproducing Kernels. , 1950 .

[17]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .