A game of prediction with expert advice

We consider the following problem. At each point of discrete time the learner must make a prediction; he is given the predictions made by a pool of experts. Each prediction and the outcome, which is disclosed after the learner has made his prediction, determine the incurred loss. It is known that, under weak regularity, the learner can ensure that his cumulative loss never exceedscL+alnn, wherecandaare some constants,nis the size of the pool, andLis the cumulative loss incurred by the best expert in the pool. We find the set of those pairs (c, a) for which this is true.

[1]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[2]  Manfred K. Warmuth,et al.  Exponentiated Gradient Versus Gradient Descent for Linear Predictors , 1997, Inf. Comput..

[3]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[4]  Yoav Freund,et al.  Predicting a binary sequence almost as well as the optimal biased coin , 2003, COLT '96.

[5]  Yoram Singer,et al.  On‐Line Portfolio Selection Using Multiplicative Updates , 1998, ICML.

[6]  Alexander Gammerman,et al.  Computational Learning and Probabilistic Reasoning , 1996 .

[7]  Erik Ordentlich,et al.  Universal portfolios with side information , 1996, IEEE Trans. Inf. Theory.

[8]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[9]  Kenji Yamanishi.  Randomized approximate aggregating strategies and their applications to prediction and discrimination , 1995, COLT '95.

[10]  Ray J. Solomonoff.  The discovery of algorithmic probability: A guide for the programming of true creativity , 1995, EuroCOLT.

[11]  David Haussler,et al.  Tight worst-case loss bounds for predicting with expert advice , 1994, EuroCOLT.

[12]  Philip M. Long,et al.  On-line learning with linear loss constraints , 1993, COLT '93.

[13]  Dean P. Foster,et al.  A Randomization Rule for Selecting Forecasts , 1993, Oper. Res..

[14]  David Haussler,et al.  How to use expert advice , 1993, STOC.

[15]  Vladimir Vovk,et al.  Universal Forecasting Algorithms , 1992, Inf. Comput..

[16]  Dean Phillips Foster.  Prediction in the Worst Case , 1991 .

[17]  Philip M. Long,et al.  On-line learning of linear functions , 1991, STOC '91.

[18]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[19]  Donald A. Martin,et al.  An Extension of Borel Determinacy , 1990, Ann. Pure Appl. Log..

[20]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[21]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[22]  Alfredo De Santis,et al.  Learning probabilistic prediction functions , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[23]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[24]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[25]  J. Rissanen.  A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[26]  Gérard D. Cohen,et al.  A nonconstructive upper bound on covering radius , 1983, IEEE Trans. Inf. Theory.

[27]  W. Feller.  An Introduction to Probability Theory and Its Applications , 1959 .

[28]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .