Online Caching Networks with Adversarial Guarantees

We study a cache network under arbitrary adversarial request arrivals. We propose a distributed online policy based on the online tabular greedy algorithm. Our distributed policy achieves sublinear (1-1/e)-regret, also in the case when update costs cannot be neglected. Numerical evaluation over several topologies supports our theoretical results and demonstrates that our algorithm outperforms state-of-art online cache algorithms.

[1]  Ronald Fagin,et al.  Asymptotic Miss Ratios over Independent References , 1977, J. Comput. Syst. Sci..

[2]  Donald F. Towsley,et al.  A utility optimization approach to network cache design , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[3]  Thrasyvoulos Spyropoulos,et al.  A Swiss Army Knife for Online Caching in Small Cell Networks , 2021, IEEE/ACM Transactions on Networking.

[4]  Lyle A. McGeoch,et al.  Competitive algorithms for on-line problems , 1988, STOC '88.

[5]  Jan Vondrák,et al.  Dependent Randomized Rounding via Exchange Properties of Combinatorial Structures , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[6]  Donald F. Towsley,et al.  On the Convergence of the TTL Approximation for an LRU Cache under Independent Stationary Request Processes , 2017, ACM Trans. Model. Perform. Evaluation Comput. Syst..

[7]  Matthew J. Streeter,et al.  An Online Algorithm for Maximizing Submodular Functions , 2008, NIPS.

[8]  Yuanyuan Li,et al.  Universally Stable Cache Networks , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[9]  Zohar S. Karnin,et al.  Multi-armed Bandits: Competing with Optimal Sequences , 2016, NIPS.

[10]  Amin Karbasi,et al.  Conditional Gradient Method for Stochastic Submodular Maximization: Closing the Gap , 2017, AISTATS.

[11]  Marco Cuturi,et al.  Computational Optimal Transport: With Applications to Data Science , 2019 .

[12]  Alexandros G. Dimakis,et al.  FemtoCaching: Wireless Content Delivery Through Distributed Caching Helpers , 2013, IEEE Transactions on Information Theory.

[13]  John Langford,et al.  Efficient Optimal Learning for Contextual Bandits , 2011, UAI.

[14]  Ronald W. Wolff,et al.  Poisson Arrivals See Time Averages , 1982, Oper. Res..

[15]  Stratis Ioannidis,et al.  Jointly Optimal Routing and Caching for Arbitrary Network Topologies , 2017, IEEE Journal on Selected Areas in Communications.

[16]  Lachlan L. H. Andrew,et al.  A tale of two metrics: simultaneous bounds on competitiveness and regret , 2013, SIGMETRICS '13.

[17]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[18]  Abhishek Sinha,et al.  Fundamental Limits on the Regret of Online Network-Caching , 2020, SIGMETRICS.

[19]  S. Shenker,et al.  Replication strategies in unstructured peer-to-peer networks , 2002, SIGCOMM '02.

[20]  Donald F. Towsley,et al.  Joint cache resource allocation and request routing for in-network caching services , 2017, Comput. Networks.

[21]  Assaf J. Zeevi,et al.  Chasing Demand: Learning and Earning in a Changing Environment , 2016, Math. Oper. Res..

[22]  Aaron Roth,et al.  Constrained Non-monotone Submodular Maximization: Offline and Secretary Algorithms , 2010, WINE.

[23]  Jan Vondrák,et al.  Maximizing a Submodular Set Function Subject to a Matroid Constraint (Extended Abstract) , 2007, IPCO.

[24]  Miguel Rio,et al.  DR-Cache: Distributed Resilient Caching with Latency Guarantees , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[25]  Andreas Krause,et al.  Online Submodular Maximization under a Matroid Constraint with Application to Learning Assignments , 2014, ArXiv.

[26]  Csaba Szepesvári,et al.  Exploration-exploitation tradeoff using variance estimates in multi-armed bandits , 2009, Theor. Comput. Sci..

[27]  Eli Upfal,et al.  Multi-Armed Bandits in Metric Spaces ∗ , 2008 .

[28]  Stratis Ioannidis,et al.  Distributed caching over heterogeneous mobile networks , 2010, SIGMETRICS '10.

[29]  Sem C. Borst,et al.  Distributed Caching Algorithms for Content Distribution Networks , 2010, 2010 Proceedings IEEE INFOCOM.

[30]  Filip Radlinski,et al.  Learning diverse rankings with multi-armed bandits , 2008, ICML '08.

[31]  John Langford,et al.  Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits , 2014, ICML.

[32]  Abhishek Sinha,et al.  Caching in Networks without Regret , 2020, ArXiv.

[33]  Yuval Filmus,et al.  Monotone Submodular Maximization over a Matroid via Non-Oblivious Local Search , 2012, SIAM J. Comput..

[34]  Sara Alouf,et al.  Performance models for hierarchy of caches: Application to modern DNS caches , 2016, Perform. Evaluation.

[35]  Florin Ciucu,et al.  Exact analysis of TTL cache networks , 2014, Perform. Evaluation.

[36]  Qiaomin Xie,et al.  Dynamic Regret of Policy Optimization in Non-stationary Environments , 2020, NeurIPS.

[37]  Rebecca Willett,et al.  Dynamical Models and tracking regret in online convex programming , 2013, ICML.

[38]  Rebecca Willett,et al.  Online Convex Optimization in Dynamic Environments , 2015, IEEE Journal of Selected Topics in Signal Processing.

[39]  T.-H. Hubert Chan,et al.  Online Submodular Maximization with Free Disposal: Randomization Beats ¼ for Partition Matroids , 2017, SODA.

[40]  Donald F. Towsley,et al.  Performance evaluation of hierarchical TTL-based cache networks , 2014, Comput. Networks.

[41]  Amin Karbasi,et al.  Projection-Free Online Optimization with Stochastic Gradient: From Convexity to Submodularity , 2018, ICML.

[42]  Elad Hazan,et al.  Introduction to Online Convex Optimization , 2016, Found. Trends Optim..

[43]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[44]  Omar Besbes,et al.  Non-Stationary Stochastic Optimization , 2013, Oper. Res..

[45]  Maxim Sviridenko,et al.  Pipage Rounding: A New Method of Constructing Algorithms with Proven Performance Guarantee , 2004, J. Comb. Optim..

[46]  David Simchi-Levi,et al.  Near-Optimal Model-Free Reinforcement Learning in Non-Stationary Episodic MDPs , 2021, ICML.

[47]  Amin Karbasi,et al.  Online Continuous Submodular Maximization: From Full-Information to Bandit Feedback , 2019, NeurIPS.

[48]  Peter Auer,et al.  Near-optimal Regret Bounds for Reinforcement Learning , 2008, J. Mach. Learn. Res..

[49]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[50]  Philippe Robert,et al.  A versatile and accurate approximation for LRU cache performance , 2012, 2012 24th International Teletraffic Congress (ITC 24).

[51]  Konstantinos Poularakis,et al.  Joint Caching and Routing in Congestible Networks of Arbitrary Topology , 2019, IEEE Internet of Things Journal.

[52]  Wei Chu,et al.  Contextual Bandits with Linear Payoff Functions , 2011, AISTATS.

[53]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[54]  Stratis Ioannidis,et al.  No-Regret Caching via Online Mirror Descent , 2021, ICC 2021 - IEEE International Conference on Communications.

[55]  Abhishek Sinha,et al.  Online Caching with Optimal Switching Regret , 2021, 2021 IEEE International Symposium on Information Theory (ISIT).

[56]  Mohamad Assaad,et al.  Energy Efficiency in Cache-Enabled Small Cell Networks With Adaptive User Clustering , 2018, IEEE Transactions on Wireless Communications.

[57]  Nikolaos Laoutaris,et al.  Meta algorithms for hierarchical Web caches , 2004, IEEE International Conference on Performance, Computing, and Communications, 2004.

[58]  Elias Koutsoupias,et al.  The k-server problem , 2009, Comput. Sci. Rev..

[59]  Jan Vondrák,et al.  Maximizing a Monotone Submodular Function Subject to a Matroid Constraint , 2011, SIAM J. Comput..

[60]  Hao Che,et al.  Hierarchical Web caching systems: modeling, design and experimental results , 2002, IEEE J. Sel. Areas Commun..

[61]  Dario Rossi,et al.  Caching performance of content centric networks under multi-path routing (and more) , 2011 .

[62]  Konstantinos Poularakis,et al.  Exploiting Caching and Multicast for 5G Wireless Networks , 2016, IEEE Transactions on Wireless Communications.

[63]  Amin Karbasi,et al.  Online Continuous Submodular Maximization , 2018, AISTATS.

[64]  Allan Borodin,et al.  An optimal on-line algorithm for metrical task system , 1992, JACM.

[65]  Haipeng Luo,et al.  Efficient Contextual Bandits in Non-stationary Worlds , 2017, COLT.

[66]  P. Jelenkovic Asymptotic approximation of the move-to-front search cost distribution and least-recently used caching fault probabilities , 1999 .

[67]  Robert E. Tarjan,et al.  Amortized efficiency of list update and paging rules , 1985, CACM.

[68]  Emilio Leonardi,et al.  Implicit Coordination of Caches in Small Cell Networks Under Unknown Popularity Profiles , 2018, IEEE Journal on Selected Areas in Communications.

[69]  Sébastien Bubeck,et al.  Convex Optimization: Algorithms and Complexity , 2014, Found. Trends Mach. Learn..

[70]  Lilian Besson,et al.  The Generalized Likelihood Ratio Test meets klUCB: an Improved Algorithm for Piece-Wise Non-Stationary Bandits , 2019, ArXiv.

[71]  Amin Karbasi,et al.  Gradient Methods for Submodular Maximization , 2017, NIPS.

[72]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[73]  Andreas Krause,et al.  Online Learning of Assignments , 2009, NIPS.

[74]  George Iosifidis,et al.  Learning to Cache With No Regrets , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[75]  Stratis Ioannidis,et al.  Intermediate Data Caching Optimization for Multi-Stage and Parallel Big Data Frameworks , 2018, 2018 IEEE 11th International Conference on Cloud Computing (CLOUD).