Defensive Forecasting for Linear Protocols

We consider a general class of forecasting protocols, called “linear protocols”, and discuss several important special cases, including multi-class forecasting. Forecasting is formalized as a game between three players: Reality, whose role is to generate objects and their labels; Forecaster, whose goal is to predict the labels; and Skeptic, who tries to make money on any lack of agreement between Forecaster’s predictions and the actual labels. Our main mathematical result is that for any continuous strategy for Skeptic in a linear protocol there exists a strategy for Forecaster that does not allow Skeptic’s capital to grow. This result is a meta-theorem that allows one to transform any constructive law of probability in a linear protocol into a forecasting strategy whose predictions are guaranteed to satisfy this law. We apply this meta-theorem to a weak law of large numbers in inner product spaces to obtain a version of the K29 prediction algorithm for linear protocols and show that this version also satisfies the attractive properties of proper calibration and resolution under a suitable choice of its kernel parameter, with no assumptions about the way the data is generated.