Bounds on Individual Risk for Log-loss Predictors

In sequential prediction with log-loss as well as density estimation with risk measured by KL divergence, one is often interested in the expected instantaneous loss, or, equivalently, the individual risk at a given xed sample size n. For Bayesian prediction and estimation methods, it is often easy to obtain bounds on the cumulative risk. Such results are based on bounding the individual sequence regret, a technique that is very well known in the COLT community. Motivated by the easiness of proofs for the cumulative risk, our open problem is to use the results on cumulative risk to prove corresponding individual-risk bounds. Background We consider sequential prediction (online learning) with log-loss (Cesa-Bianchi and Lugosi, 2006). In each iteration n = 1; 2;:::, after observing a sequence of past outcomes x n = x1;x2;:::;xn 2 X n , a prediction strategy assigns a probability distribution on X , denoted ^ P ( j