A game of prediction with expert advice

We consider the following problem. At each point of discrete time the learner must make a prediction; he is given the predictions made by a pool of experts. Each prediction and the outcome, which is disclosed after the learner has made his prediction, determine the incurred loss. It is known that, under weak regularity, the learner can ensure that his cumulative loss never exceedscL+alnn, wherecandaare some constants,nis the size of the pool, andLis the cumulative loss incurred by the best expert in the pool. We find the set of those pairs (c, a) for which this is true.

[1]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[2]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[3]  Gérard D. Cohen,et al.  A nonconstructive upper bound on covering radius , 1983, IEEE Trans. Inf. Theory.

[4]  J. Rissanen A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[5]  A. P. Dawid,et al.  Present position and potential developments: some personal views , 1984 .

[6]  N. Littlestone Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[7]  Alfredo De Santis,et al.  Learning probabilistic prediction functions , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[8]  Donald A. Martin,et al.  An Extension of Borel Determinacy , 1990, Ann. Pure Appl. Log..

[9]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[10]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[11]  Dean Phillips Foster Prediction in the Worst Case , 1991 .

[12]  Philip M. Long,et al.  On-line learning of linear functions , 1991, STOC '91.

[13]  Vladimir Vovk,et al.  Universal Forecasting Algorithms , 1992, Inf. Comput..

[14]  David Haussler,et al.  How to use expert advice , 1993, STOC.

[15]  Dean P. Foster,et al.  A Randomization Rule for Selecting Forecasts , 1993, Oper. Res..

[16]  Philip M. Long,et al.  On-line learning with linear loss constraints , 1993, COLT '93.

[17]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[18]  David Haussler,et al.  Tight worst-case loss bounds for predicting with expert advice , 1994, EuroCOLT.

[19]  Ray J. Solomonoff The discovery of algorithmic probability: A guide for the programming of true creativity , 1995, EuroCOLT.

[20]  Kenji Yamanishi Randomized approximate aggregating strategies and their applications to prediction and discrimination , 1995, COLT '95.

[21]  Mark Herbster,et al.  Tracking the Best Expert , 1995, Machine-mediated learning.

[22]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[23]  Erik Ordentlich,et al.  Universal portfolios with side information , 1996, IEEE Trans. Inf. Theory.

[24]  Yoav Freund,et al.  Predicting a binary sequence almost as well as the optimal biased coin , 2003, COLT '96.

[25]  Alexander Gammerman,et al.  Computational Learning and Probabilistic Reasoning , 1996 .

[26]  Yoram Singer,et al.  On‐Line Portfolio Selection Using Multiplicative Updates , 1998, ICML.

[27]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[28]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[29]  Manfred K. Warmuth,et al.  Exponentiated Gradient Versus Gradient Descent for Linear Predictors , 1997, Inf. Comput..

[30]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.