Defensive forecasting for optimal prediction with expert advice

The method of defensive forecasting is applied to the problem of prediction with expert advice for binary outcomes. It turns out that defensive forecasting is not only competitive with the Aggregating Algorithm but also handles the case of “second-guessing” experts, whose advice depends on the learner’s prediction; this paper assumes that the dependence on the learner’s prediction is continuous.

[1]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[2]  Marcus Hutter,et al.  Adaptive Online Prediction by Following the Perturbed Leader , 2005, J. Mach. Learn. Res..

[3]  Vladimir Vovk,et al.  A game of prediction with expert advice , 1995, COLT '95.

[4]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[5]  V. Vovk.  Competitive On‐line Statistics , 2001 .

[6]  Santosh S. Vempala,et al.  Efficient algorithms for online decision problems , 2005, J. Comput. Syst. Sci..

[7]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[8]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[9]  David Haussler,et al.  Sequential Prediction of Individual Sequences Under General Loss Functions , 1998, IEEE Trans. Inf. Theory.

[10]  Gábor Lugosi,et al.  Learning correlated equilibria in games with compact sets of strategies , 2007, Games Econ. Behav..

[11]  Manfred K. Warmuth,et al.  Exponentiated Gradient Versus Gradient Descent for Linear Predictors , 1997, Inf. Comput..

[12]  Philip M. Long,et al.  Worst-case quadratic loss bounds for prediction using linear functions and gradient descent , 1996, IEEE Trans. Neural Networks.

[13]  G. Shafer,et al.  Probability and Finance: It's Only a Game! , 2001 .

[14]  Vladimir Vovk,et al.  Predictions as Statements and Decisions , 2006, COLT.

[15]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[16]  James Hannan,et al.  4. APPROXIMATION TO RAYES RISK IN REPEATED PLAY , 1958 .

[17]  Vladimir Vovk,et al.  Derandomizing Stochastic Prediction Strategies , 1997, COLT '97.

[18]  Akimichi Takemura,et al.  Defensive Forecasting , 2005, AISTATS.

[19]  Péter Gács,et al.  Uniform test of algorithmic randomness over a general space , 2003, Theor. Comput. Sci..

[20]  Philip M. Long,et al.  WORST-CASE QUADRATIC LOSS BOUNDS FOR ON-LINE PREDICTION OF LINEAR FUNCTIONS BY GRADIENT DESCENT , 1993 .

[21]  Manfred K. Warmuth,et al.  Averaging Expert Predictions , 1999, EuroCOLT.

[22]  Akimichi Takemura,et al.  Defensive Forecasting for Linear Protocols , 2005, ALT.