Following the Leader and Fast Rates in Linear Prediction: Curved Constraint Sets and Other Regularities

The follow the leader (FTL) algorithm, perhaps the simplest of all online learning algorithms, is known to perform well when the loss functions it is used on are positively curved. In this paper we ask whether there are other "lucky" settings when FTL achieves sublinear, "small" regret. In particular, we study the fundamental problem of linear prediction over a non-empty convex, compact domain. Amongst other results, we prove that the curvature of the boundary of the domain can act as if the losses were curved: In this case, we prove that as long as the mean of the loss vectors have positive lengths bounded away from zero, FTL enjoys a logarithmic growth rate of regret, while, e.g., for polyhedral domains and stochastic data it enjoys finite expected regret. Building on a previously known meta-algorithm, we also get an algorithm that simultaneously enjoys the worst-case guarantees and the bound available for FTL.

[1]  Boris Polyak,et al.  Constrained minimization methods , 1966 .

[2]  Neri Merhav,et al.  Universal sequential learning and decision from individual data sequences , 1992, COLT '92.

[3]  R. Schneider.  Convex Bodies: The Brunn–Minkowski Theory: Minkowski addition , 1993 .

[4]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[5]  E. Polovinkin.  Strongly convex analysis , 1996 .

[6]  Fabio Stella,et al.  Stochastic Nonstationary Optimization for Finding Universal Portfolios , 2000, Ann. Oper. Res..

[7]  A. Pressley.  Elementary Differential Geometry , 2000 .

[8]  Claudio Gentile,et al.  On the generalization ability of on-line learning algorithms , 2001, IEEE Transactions on Information Theory.

[9]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[10]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[11]  Peter L. Bartlett,et al.  Adaptive Online Gradient Descent , 2007, NIPS.

[12]  Ambuj Tewari,et al.  Optimal Stragies and Minimax Lower Bounds for Online Convex Games , 2008, COLT.

[13]  Sham M. Kakade,et al.  Mind the Duality Gap: Logarithmic regret algorithms for online optimization , 2008, NIPS.

[14]  H. Brendan McMahan,et al.  Follow-the-Regularized-Leader and Mirror Descent: Equivalence Theorems and Implicit Updates , 2010, arXiv.org.

[15]  Yurii Nesterov,et al.  Generalized Power Method for Sparse Principal Component Analysis , 2008, J. Mach. Learn. Res..

[16]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[17]  Claudio Gentile,et al.  Beyond Logarithmic Bounds in Online Learning , 2012, AISTATS.

[18]  Karthik Sridharan,et al.  Online Learning with Predictable Sequences , 2012, COLT.

[19]  Alessandro Lazaric,et al.  Exploiting easy data in online optimization , 2014, NIPS.

[20]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[21]  Karthik Sridharan,et al.  Adaptive Online Learning , 2015, NIPS.

[22]  Elad Hazan,et al.  Faster Rates for the Frank-Wolfe Method over Strongly-Convex Sets , 2014, ICML.

[23]  Wojciech Kot lowski.  Minimax strategy for prediction with expert advice under stochastic assumptions , 2015 .

[24]  Mark D. Reid,et al.  Fast rates in statistical and online learning , 2015, J. Mach. Learn. Res..

[25]  Tor Lattimore,et al.  Following the Leader and Fast Rates in Linear Prediction: Curved Constraint Sets and Other Regularities , 2016, NIPS.

[26]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .