Affine-Invariant Online Optimization and the Low-rank Experts Problem

We present a new affine-invariant optimization algorithm called Online Lazy Newton. The regret of Online Lazy Newton is independent of conditioning: the algorithm's performance depends on the best possible preconditioning of the problem in retrospect and on its \emph{intrinsic} dimensionality. As an application, we show how Online Lazy Newton can be used to achieve an optimal regret of order $\sqrt{rT}$ for the low-rank experts problem, improving by a $\sqrt{r}$ factor over the previously best known bound and resolving an open problem posed by Hazan et al (2016).

[1]  Shie Mannor,et al.  Online Learning with Many Experts , 2017, ArXiv.

[2]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[3]  Elad Hazan,et al.  Better Algorithms for Benign Bandits , 2009, J. Mach. Learn. Res..

[4]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[5]  V. Vovk.  Competitive On‐line Statistics , 2001 .

[6]  Ashok Cutkosky,et al.  Online Learning Without Prior Information , 2017, COLT.

[7]  Manfred K. Warmuth,et al.  Relative Loss Bounds for On-Line Density Estimation with the Exponential Family of Distributions , 1999, Machine Learning.

[8]  Wouter M. Koolen,et al.  Follow the leader if you can, hedge if you must , 2013, J. Mach. Learn. Res..

[9]  Elad Hazan,et al.  On Stochastic and Worst-case Models for Investing , 2009, NIPS.

[10]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[11]  Claudio Gentile,et al.  A Second-Order Perceptron Algorithm , 2002, SIAM J. Comput..

[12]  Ohad Shamir,et al.  Localization and Adaptation in Online Learning , 2013, AISTATS.

[13]  Aditya Gopalan,et al.  Online Learning for Structured Loss Spaces , 2017, AAAI.

[14]  Rong Jin,et al.  25th Annual Conference on Learning Theory Online Optimization with Gradual Variations , 2022 .

[15]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[16]  Wouter M. Koolen,et al.  MetaGrad: Multiple Learning Rates in Online Learning , 2016, NIPS.

[17]  Elad Hazan,et al.  Extracting certainty from uncertainty: regret bounded by variation in costs , 2008, Machine Learning.

[18]  Kfir Y. Levy,et al.  Fast Rates for Exp-concave Empirical Risk Minimization , 2015, NIPS.

[19]  Haipeng Luo,et al.  Efficient Second Order Online Learning by Sketching , 2016, NIPS.

[20]  Roi Livni,et al.  Online Learning with Low Rank Experts , 2016, COLT.

[21]  Karthik Sridharan,et al.  Online Learning with Predictable Sequences , 2012, COLT.

[22]  Santosh S. Vempala,et al.  Efficient algorithms for online decision problems , 2005, Journal of computer and system sciences (Print).

[23]  Matthew J. Streeter,et al.  Less Regret via Online Conditioning , 2010, ArXiv.

[24]  Karthik Sridharan,et al.  ZigZag: A New Approach to Adaptive Online Learning , 2017, COLT.