Calibrated Forecasts: The Minimax Proof

A formal write-up of the simple proof (1995) of the existence of calibrated forecasts by the minimax theorem, which moreover shows that N 3 periods suffice to guarantee a 1 /N calibration error. Consider a weather forecaster who announces each day a probability p that there will be rain tomorrow. The forecaster is said to be calibrated if, for each forecast p that is used, the relative frequency of rainy days among the days where the forecast was p is, in the long run, (close to) p . The surprising result of Foster and Vohra (1998) is that calibration can be guaranteed , no matter what the weather will be. 1 A simple proof of this result, based on the minimax theorem , was provided by the author in 1995. 2 We provide here a formal write-up of this proof, which moreover shows that an expected calibration error of size ε is guaranteed after 1 /ε 3 periods. For each period (day) t = 1 , 2 , ..., let a t ∈ { 0 , 1 } be the weather , with 1 for rain and 0 for no rain, and let c t ∈ [0 , 1] be the forecast . We will let our