On sequential prediction of individual sequences relative to a set of experts

We investigate sequential randomized prediction of an arbitrary sequence taking values from a binary alphabet. The goal of the predictor is to minimize his Hamming loss relative to the loss of the best “expert” in a fixed, possibly infinite, set of experts. We point out a surprisingly close connection between the prediction problem and empirical process theory. Using ideas and results from empirical process theory, we show upper and lower bounds on the minimax relative loss in terms of the geometry of the class of experts. As a main example, we determine the exact order of magnitude of the minimax relative loss for the class of Markov experts. Furthermore, in the special case of static experts, we completely characterize the minimax relative loss in terms of the maximal deviation of an associated Rademacher process.