Online Optimization with Predictions and Non-convex Losses

We study online optimization in a setting where an online learner seeks to optimize a per-round hitting cost, which may be non-convex, while incurring a movement cost when changing actions between rounds. We ask:under what general conditions is it possible for an online learner to leverage predictions of future cost functions in order to achieve near-optimal costs? Prior work has provided near-optimal online algorithms for specific combinations of assumptions about hitting and switching costs, but no general results are known. In this work, we give two general sufficient conditions that specify a relationship between the hitting and movement costs which guarantees that a new algorithm, Synchronized Fixed Horizon Control (SFHC), achieves a 1+O(1/w) competitive ratio, where w is the number of predictions available to the learner. Our conditions do not require the cost functions to be convex, and we also derive competitive ratio results for non-convex hitting and movement costs. Our results provide the first constant, dimension-free competitive ratio for online non-convex optimization with movement costs. We also give an example of a natural problem, Convex Body Chasing (CBC), where the sufficient conditions are not satisfied and prove that no online algorithm can have a competitive ratio that converges to 1.

[1]  Mark Sellke.  Chasing Convex Bodies Optimally , 2020, SODA.

[2]  Andrew Tomkins,et al.  A polylog(n)-competitive algorithm for metrical task systems , 1997, STOC '97.

[3]  Lachlan L. H. Andrew,et al.  Geographical load balancing with renewables , 2011, PERV.

[4]  Na Li,et al.  Online Optimization With Predictions and Switching Costs: Fast Algorithms and the Fundamental Limit , 2018, IEEE Transactions on Automatic Control.

[5]  Jean-Yves Audibert,et al.  Regret Bounds and Minimax Policies under Partial Monitoring , 2010, J. Mach. Learn. Res..

[6]  David Ardia,et al.  Differential Evolution (DEoptim) for Non-Convex Portfolio Optimization , 2010 .

[7]  Adam Wierman,et al.  An Online Algorithm for Smoothed Regression and LQR Control , 2018, AISTATS.

[8]  Yisong Yue,et al.  A Decision Tree Framework for Spatiotemporal Sequence Prediction , 2015, KDD.

[9]  Adam Wierman,et al.  Online convex optimization with ramp constraints , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[10]  Lachlan L. H. Andrew,et al.  A tale of two metrics: simultaneous bounds on competitiveness and regret , 2013, SIGMETRICS '13.

[11]  Anupam Gupta,et al.  Chasing Convex Bodies with Linear Competitive Ratio , 2019, SODA.

[12]  Adam Wierman,et al.  Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent , 2018, COLT.

[13]  Stephen P. Boyd,et al.  Online convex optimization-based algorithm for thermal management of MPSoCs , 2010, GLSVLSI '10.

[14]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[15]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[16]  Kirk Pruhs,et al.  Chasing Convex Bodies and Functions , 2016, LATIN.

[17]  Georgios B. Giannakis,et al.  An Online Convex Optimization Approach to Real-Time Energy Pricing for Demand Response , 2017, IEEE Transactions on Smart Grid.

[18]  Adam Wierman,et al.  Beyond Online Balanced Descent: An Optimal Algorithm for Smoothed Online Optimization , 2019, NeurIPS.

[19]  Sébastien Bubeck,et al.  Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems , 2012, Found. Trends Mach. Learn..

[20]  Lachlan L. H. Andrew,et al.  Dynamic Right-Sizing for Power-Proportional Data Centers , 2011, IEEE/ACM Transactions on Networking.

[21]  Yin Tat Lee,et al.  A Nearly-Linear Bound for Chasing Nested Convex Bodies , 2018, SODA.

[22]  Sanjo Zlobec.  Jensen's inequality for nonconvex functions ∗ , 2004 .

[23]  Yin Tat Lee,et al.  Chasing Nested Convex Bodies Nearly Optimally , 2018, SODA.

[24]  Lyle A. McGeoch,et al.  Competitive Algorithms for Server Problems , 1990, J. Algorithms.

[25]  Nikhil Bansal,et al.  Potential-Function Proofs for First-Order Methods , 2017, ArXiv.

[26]  Peter L. Bartlett,et al.  Improved Generalization Through Explicit Optimization of Margins , 2000, Machine Learning.

[27]  Adam Wierman,et al.  Using Predictions in Online Optimization: Looking Forward with an Eye on the Past , 2016, SIGMETRICS.

[28]  Lachlan L. H. Andrew,et al.  Online Convex Optimization Using Predictions , 2015, SIGMETRICS.

[29]  D. Hunter.  Portfolio optimization with conditional value-at-risk objective and constraints , 2002 .

[30]  Avrim Blum,et al.  On-line Learning and the Metrical Task System Problem , 1997, COLT '97.

[31]  James R. Lee,et al.  k-server via multiscale entropic regularization , 2017, STOC.

[32]  Xiaojun Lin,et al.  Retracted on December 2, 2020: On the Value of Look-Ahead in Competitive Online Convex Optimization , 2019, SIGMETRICS.

[33]  Adam Wierman,et al.  Thinking Fast and Slow , 2017, SIGMETRICS Perform. Evaluation Rev..

[34]  Na Li,et al.  Using Predictions in Online Optimization with Switching Costs: A Fast Algorithm and A Fundamental Limit , 2018, 2018 Annual American Control Conference (ACC).

[35]  Zhenhua Liu,et al.  Online Optimization in Cloud Resource Provisioning , 2019, Proc. ACM Meas. Anal. Comput. Syst..

[36]  Prateek Jain,et al.  Non-convex Optimization for Machine Learning , 2017, Found. Trends Mach. Learn..

[37]  Wing Shing Wong,et al.  An Optimal Algorithm for Online Non-Convex Learning , 2019, PERV.

[38]  Joseph Naor,et al.  k-Servers with a Smile: Online Algorithms via Projections , 2018, SODA.

[39]  Kirk Pruhs,et al.  A 2-Competitive Algorithm For Online Convex Optimization With Switching Costs , 2015, APPROX-RANDOM.

[40]  Kevin Schewior,et al.  A Tight Lower Bound for Online Convex Optimization with Switching Costs , 2017, WAOA.

[41]  Elad Hazan,et al.  Introduction to Online Convex Optimization , 2016, Found. Trends Optim..

[42]  Gustavo de Veciana,et al.  Jointly optimizing multi-user rate adaptation for video transport over wireless systems: Mean-fairness-variability tradeoffs , 2012, 2012 Proceedings IEEE INFOCOM.

[43]  Allan Borodin,et al.  An optimal on-line algorithm for metrical task system , 1992, JACM.

[44]  Steven H. Low,et al.  Convex Relaxation of Optimal Power Flow—Part I: Formulations and Equivalence , 2014, IEEE Transactions on Control of Network Systems.

[45]  C. Lee Giles,et al.  Nonconvex Online Support Vector Machines , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Haipeng Luo,et al.  Improved Path-length Regret Bounds for Bandits , 2019, COLT.

[47]  Lachlan L. H. Andrew,et al.  Online algorithms for geographical load balancing , 2012, 2012 International Green Computing Conference (IGCC).

[48]  L H AndrewLachlan,et al.  Dynamic right-sizing for power-proportional data centers , 2013 .

[49]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[50]  Steven H. Low,et al.  Convex Relaxation of Optimal Power Flow—Part II: Exactness , 2014, IEEE Transactions on Control of Network Systems.