Transient and steady-state regime of a family of list-based cache replacement algorithms

We study the performance of a family of cache replacement algorithms. The cache is decomposed into lists. Some of these lists can be virtual in the sense that only meta-data are stored in those lists. An item enters the cache via the first list and jumps to the next list whenever a hit on it occurs. The classical policies FIFO, RANDOM, CLIMB, and its hybrids are obtained as special cases. We present explicit expressions for the cache content distribution and miss probability under the IRM model. We develop an algorithm with a time complexity that is polynomial in the cache size and linear in the number of items to compute the exact miss probability. We introduce lower and upper bounds on the latter that can be computed in a time that is linear in the cache size times the number of items. We introduce a mean-field model to approximate the transient behavior of the miss probability and prove that this model becomes exact as the cache size and number of items go to infinity. We show that the set of ODEs associated to the mean-field model has a unique fixed point that can be used to approximate the miss probability in case the exact computation is too time consuming. Using this approximation, we provide guidelines on how to select a replacement algorithm within the family considered such that a good trade-off is achieved between the cache reactivity and its steady-state hit probability. We simulate these cache replacement algorithms on traces of real data and show that they can outperform LRU. Finally, we also disprove the well-known conjecture that the CLIMB algorithm is the optimal finite-memory replacement algorithm under the IRM model.

[1]  Florin Ciucu,et al.  Exact analysis of TTL cache networks , 2014, Perform. Evaluation.

[2]  Naoto Miyoshi,et al.  Fluid limit analysis of FIFO and RR caching for independent reference models , 2012, Perform. Evaluation.

[3]  Dennis Shasha,et al.  2Q: A Low Overhead High Performance Buffer Management Replacement Algorithm , 1994, VLDB.

[4]  Michele Garetto,et al.  A unified approach to the performance analysis of caching systems , 2014, INFOCOM.

[5]  Benny Van Houdt,et al.  Asymptotically Exact TTL-Approximations of the Cache Replacement Algorithms LRU(m) and h-LRU , 2016, 2016 28th International Teletraffic Congress (ITC 28).

[6]  Asit Dan,et al.  An approximate analysis of the LRU and FIFO buffer replacement schemes , 1990, SIGMETRICS '90.

[7]  Paolo Giaccone,et al.  Temporal locality in today's content caching: why it matters and how to model it , 2013, CCRV.

[8]  Takayuki Osogami,et al.  Analysis of Page Replacement Policies in the Fluid Limit , 2010, Oper. Res..

[9]  Domenico Ferrari,et al.  Two-Level Replacement Decisions in Paging Stores , 1983, IEEE Transactions on Computers.

[10]  P. Jelenkovic.  Asymptotic approximation of the move-to-front search cost distribution and least-recently used caching fault probabilities , 1999 .

[11]  Daniel S. Hirschberg,et al.  Self-organizing linear search , 1985, CSUR.

[12]  Michael Zink,et al.  Characteristics of YouTube network traffic at a campus network - Measurements, models, and implications , 2009, Comput. Networks.

[13]  Roy D. Yates,et al.  A Framework for Uplink Power Control in Cellular Radio Systems , 1995, IEEE J. Sel. Areas Commun..

[14]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[15]  Donald F. Towsley,et al.  Analysis of TTL-based cache networks , 2012, 6th International ICST Conference on Performance Evaluation Methodologies and Tools.

[16]  Song Jiang,et al.  LIRS: an efficient low inter-reference recency set replacement policy to improve buffer cache performance , 2002, SIGMETRICS '02.

[17]  Alberto Gandolfi,et al.  LRU is better than FIFO under the independent reference model , 1992, Journal of Applied Probability.

[18]  Benny Van Houdt,et al.  Transient and steady-state regime of a family of list-based cache replacement algorithms , 2015, Queueing Systems.

[19]  T. Kurtz.  Approximation of Population Processes , 1987 .

[20]  Hao Che,et al.  Hierarchical Web caching systems: modeling, design and experimental results , 2002, IEEE J. Sel. Areas Commun..

[21]  Erol Gelenbe,et al.  A Unified Approach to the Evaluation of a Class of Replacement Algorithms , 1973, IEEE Transactions on Computers.

[22]  Donald F. Towsley,et al.  Properties of the Miss Ratio for a 2-Level Storage Model with LRU or FIFO Replacement Strategy and Independent References , 1993, IEEE Trans. Computers.

[23]  Armand M. Makowski,et al.  Comparing strength of locality of reference - popularity, majorization, and some folk theorems , 2004, IEEE INFOCOM 2004.

[24]  Donald F. Towsley,et al.  Approximate Models for General Cache Networks , 2010, 2010 Proceedings IEEE INFOCOM.

[25]  Hari Balakrishnan,et al.  Modeling TTL-based Internet caches , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[26]  Jean-Yves Le Boudec,et al.  A class of mean field interaction models for computer and communication systems , 2008, Perform. Evaluation.

[27]  Li Fan,et al.  Web caching and Zipf-like distributions: evidence and implications , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[28]  Philippe Robert,et al.  A versatile and accurate approximation for LRU cache performance , 2012, 2012 24th International Teletraffic Congress (ITC 24).

[29]  E. G. Coffman,et al.  Stochastic Analysis of Computer Storage , 1987 .

[30]  Yakov A. Kogan,et al.  Some Results on Distribution-Free Analysis of Paging Algorithms , 1976, IEEE Transactions on Computers.

[31]  Ronald Fagin,et al.  Efficient Calculation of Expected Miss Ratios in the Independent Reference Model , 1978, SIAM J. Comput..

[32]  W. Frank King,et al.  Analysis of Demand Paging Algorithms , 1971, IFIP Congress.

[33]  C. Fortuin,et al.  Correlation inequalities on some partially ordered sets , 1971 .

[34]  Armand M. Makowski,et al.  The output of a cache under the independent reference model: where did the locality of reference go? , 2004, SIGMETRICS '04/Performance '04.

[35]  Nikolaos Laoutaris,et al.  The LCD interconnection of LRU caches and its analysis , 2006, Perform. Evaluation.

[36]  Massimo Gallo,et al.  Performance evaluation of the random replacement policy for networks of caches , 2014, Perform. Evaluation.

[37]  Jean-Yves Le Boudec,et al.  The stationary behaviour of fluid limits of reversible processes is concentrated on stationary points , 2010, Networks Heterog. Media.

[38]  Emilio Leonardi,et al.  Least recently used caches under the Shot Noise Model , 2014, 2015 IEEE Conference on Computer Communications (INFOCOM).

[39]  David Tse,et al.  Probabilistic methods for web caching , 2001, Perform. Evaluation.