Understand Dynamic Regret with Switching Cost for Online Decision Making

As a metric to measure the performance of an online method, dynamic regret with switching cost has drawn much attention for online decision making problems. Although the sublinear regret has been provided in much previous research, we still have little knowledge about the relation between the dynamic regret and the switching cost. In the article, we investigate the relation for two classic online settings: Online Algorithms (OA) and Online Convex Optimization (OCO). We provide a new theoretical analysis framework that shows an interesting observation; that is, the relation between the switching cost and the dynamic regret is different for settings of OA and OCO. Specifically, the switching cost has significant impact on the dynamic regret in the setting of OA. But it does not have an impact on the dynamic regret in the setting of OCO. Furthermore, we provide a lower bound of regret for the setting of OCO, which is same with the lower bound in the case of no switching cost. It shows that the switching cost does not change the difficulty of online decision making problems in the setting of OCO.

[1]  Qian Wang,et al.  Differentially Private Distributed Online Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[2]  Jay H. Lee,et al.  Model predictive control: past, present and future , 1999 .

[3]  Steven C. H. Hoi,et al.  Online portfolio selection: A survey , 2012, CSUR.

[4]  Cédric Archambeau,et al.  Adaptive Algorithms for Online Convex Optimization with Long-term Constraints , 2015, ICML.

[5]  Avrim Blum,et al.  On-line Learning and the Metrical Task System Problem , 1997, COLT '97.

[6]  Adi Rosén,et al.  On Online Algorithms with Advice for the k-Server Problem , 2011, WAOA.

[7]  Benjamin F. Hobbs,et al.  Dynamic oligopolistic competition on an electric power network with ramping costs and joint sales constraints , 2008 .

[8]  Qi Zhang,et al.  Dynamic Service Placement in Geographically Distributed Clouds , 2013, IEEE J. Sel. Areas Commun..

[9]  Sébastien Bubeck,et al.  Introduction to Online Optimization , 2011 .

[10]  Bin Li,et al.  Confidence Weighted Mean Reversion Strategy for Online Portfolio Selection , 2011, TKDD.

[11]  Xiaobo Li,et al.  Online Learning with Non-Convex Losses and Non-Stationary Regret , 2018, AISTATS.

[12]  Shui Yu,et al.  Differential Privacy Oriented Distributed Online Learning for Mobile Social Video Prefetching , 2019, IEEE Transactions on Multimedia.

[13]  Rong Jin,et al.  Dynamic Regret of Strongly Adaptive Methods , 2017, ICML.

[14]  Jinfeng Yi,et al.  Improved Dynamic Regret for Non-degenerate Functions , 2016, NIPS.

[15]  Marc Teboulle,et al.  Mirror descent and nonlinear projected subgradient methods for convex optimization , 2003, Oper. Res. Lett..

[16]  Liang Wang,et al.  Improving Advertisement Recommendation by Enriching User Browser Cookie Attributes , 2016, CIKM.

[17]  Adi Rosén,et al.  On Online Algorithms with Advice for the k-Server Problem , 2011, Theory of Computing Systems.

[18]  Rong Jin,et al.  25th Annual Conference on Learning Theory Online Optimization with Gradual Variations , 2022 .

[19]  Aryan Mokhtari,et al.  Optimization in Dynamic Environments : Improved Regret Rates for Strongly Convex Problems , 2016 .

[20]  Qing Ling,et al.  An Online Convex Optimization Approach to Proactive Network Resource Allocation , 2017, IEEE Transactions on Signal Processing.

[21]  Michael R. Lyu,et al.  Efficient online learning for multitask feature selection , 2013, TKDD.

[22]  Joseph Naor,et al.  Metrical Task Systems and the k-Server Problem on HSTs , 2010, ICALP.

[23]  Lijun Zhang,et al.  Adaptive Online Learning in Dynamic Environments , 2018, NeurIPS.

[24]  Lachlan L. H. Andrew,et al.  Dynamic Right-Sizing for Power-Proportional Data Centers , 2011, IEEE/ACM Transactions on Networking.

[25]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[26]  Adam Wierman,et al.  Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent , 2018, COLT.

[27]  Kevin Schewior,et al.  A Tight Lower Bound for Online Convex Optimization with Switching Costs , 2017, WAOA.

[28]  Rebecca Willett,et al.  Dynamical Models and tracking regret in online convex programming , 2013, ICML.

[29]  Koby Crammer,et al.  Online Classification on a Budget , 2003, NIPS.

[30]  Rebecca Willett,et al.  Online Convex Optimization in Dynamic Environments , 2015, IEEE Journal of Selected Topics in Signal Processing.

[31]  Elad Hazan,et al.  Introduction to Online Convex Optimization , 2016, Found. Trends Optim..

[32]  James R. Lee,et al.  k-server via multiscale entropic regularization , 2017, STOC.

[33]  James R. Lee,et al.  Fusible HSTs and the Randomized k-Server Conjecture , 2017, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[34]  Shie Mannor,et al.  Online Classification with Specificity Constraints , 2010, NIPS.

[35]  James R. Lee,et al.  Metrical task systems on trees via mirror descent and unfair gluing , 2018, SODA.

[36]  Ji Liu,et al.  Proximal Online Gradient Is Optimum for Dynamic Regret: A General Lower Bound , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Joseph Naor,et al.  Unified Algorithms for Online Learning and Competitive Analysis , 2012, COLT.

[38]  Sébastien Bubeck,et al.  Convex Optimization: Algorithms and Complexity , 2014, Found. Trends Mach. Learn..

[39]  Na Li,et al.  Online Optimization With Predictions and Switching Costs: Fast Algorithms and the Fundamental Limit , 2018, IEEE Transactions on Automatic Control.

[40]  Shahin Shahrampour,et al.  Online Optimization : Competing with Dynamic Comparators , 2015, AISTATS.

[41]  Omar Besbes,et al.  Non-Stationary Stochastic Optimization , 2013, Oper. Res..

[42]  Adam Wierman,et al.  Using Predictions in Online Optimization: Looking Forward with an Eye on the Past , 2016, SIGMETRICS.

[43]  Lachlan L. H. Andrew,et al.  Online Convex Optimization Using Predictions , 2015, SIGMETRICS.

[44]  Minghua Chen,et al.  Simple and effective dynamic provisioning for power-proportional data centers , 2011, 2012 46th Annual Conference on Information Sciences and Systems (CISS).

[45]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[46]  Lachlan L. H. Andrew,et al.  Online optimization with switching cost , 2012, PERV.

[47]  András György,et al.  Shifting Regret, Mirror Descent, and Matrices , 2016, ICML.

[48]  Na Li,et al.  Using Predictions in Online Optimization with Switching Costs: A Fast Algorithm and A Fundamental Limit , 2018, 2018 Annual American Control Conference (ACC).

[49]  Sun.  Online Ensemble Learning of Data Streams with Gradually Evolved Classes , 2016 .

[50]  Jinfeng Yi,et al.  Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient , 2016, ICML.

[51]  Peter L. Bartlett,et al.  A Regularization Approach to Metrical Task Systems , 2010, ALT.

[52]  Hamed Mohsenian Rad,et al.  Exploring smart grid and data center interactions for electric power load balancing , 2014, PERV.

[53]  Lachlan L. H. Andrew,et al.  A tale of two metrics: simultaneous bounds on competitiveness and regret , 2013, SIGMETRICS '13.