Load Imbalance and Caching Performance of Sharded Systems

Sharding is a method for allocating data items to nodes of a distributed caching or storage system based on the result of a hash function computed on the item’s identifier. It is ubiquitously used in key-value stores, CDNs and many other applications. Despite considerable work that has focused on the design and implementation of such systems, there is limited understanding of their performance in realistic operational conditions from a theoretical standpoint. In this paper we fill this gap by providing a thorough modeling of sharded caching systems, focusing particularly on load balancing and caching performance aspects. Our analysis provides important insights that can be applied to optimize the design and configuration of sharded caching systems.

[1]  Martin Raab,et al.  "Balls into Bins" - A Simple and Tight Analysis , 1998, RANDOM.

[2]  George Pavlou,et al.  Understanding sharded caching systems , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[3]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[4]  Ronald W. Wolff,et al.  Poisson Arrivals See Time Averages , 1982, Oper. Res..

[5]  Dario Rossi,et al.  Analyzing cacheable traffic in isp access networks for micro cdn applications via content-centric networking , 2014, ICN '14.

[6]  Guocong Quan,et al.  Asymptotic Miss Ratio of LRU Caching with Consistent Hashing , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[7]  Peter J. Denning,et al.  Operating Systems Theory , 1973 .

[8]  Philippe Robert,et al.  A versatile and accurate approximation for LRU cache performance , 2012, 2012 24th International Teletraffic Congress (ITC 24).

[9]  Dong Zhou,et al.  Scaling Up Clustered Network Appliances with ScaleBricks , 2015, SIGCOMM.

[10]  Tony Tung,et al.  Scaling Memcache at Facebook , 2013, NSDI.

[11]  Bin Fan,et al.  Small cache, big effect: provable load balancing for randomly partitioned cluster services , 2011, SoCC.

[12]  Ronald Fagin,et al.  Asymptotic Miss Ratios over Independent References , 1977, J. Comput. Syst. Sci..

[13]  Gade Krishna,et al.  A scalable peer-to-peer lookup protocol for Internet applications , 2012 .

[14]  P. Jelenkovic.  Asymptotic approximation of the move-to-front search cost distribution and least-recently used caching fault probabilities , 1999 .

[15]  Amar Phanishayee,et al.  FAWN: a fast array of wimpy nodes , 2009, SOSP '09.

[16]  Song Jiang,et al.  Workload analysis of a large-scale key-value store , 2012, SIGMETRICS '12.

[17]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[18]  Bruce M. Maggs,et al.  Algorithmic Nuggets in Content Delivery , 2015, CCRV.

[19]  David Thaler,et al.  Using name-based mappings to increase hit rates , 1998, TNET.

[20]  Hao Che,et al.  Hierarchical Web caching systems: modeling, design and experimental results , 2002, IEEE J. Sel. Areas Commun..

[21]  Li Fan,et al.  Web caching and Zipf-like distributions: evidence and implications , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[22]  Robbert van Renesse,et al.  Characterizing Load Imbalance in Real-World Networked Caches , 2014, HotNets.

[23]  James Allen Fill,et al.  Limits and Rates of Convergence for the Distribution of Search Cost Under the Move-to-Front Rule , 1996, Theor. Comput. Sci..

[24]  Predrag R. Jelenkovic,et al.  Characterizing the miss sequence of the LRU cache , 2008, PERV.

[25]  Yi Sun,et al.  Trace-Driven Analysis of ICN Caching Algorithms on Video-on-Demand Workloads , 2014, CoNEXT.

[26]  Bruce M. Maggs,et al.  Less pain, most of the gain: incrementally deployable ICN , 2013, SIGCOMM.

[27]  Diego Perino,et al.  Caesar: A content router for high-speed forwarding on content names , 2012, 2014 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS).

[28]  Xiaozhou Li,et al.  Be Fast, Cheap and in Control with SwitchKV , 2016, NSDI.

[29]  Udi Wieder,et al.  Hashing, Load Balancing and Multiple Choice , 2017, Found. Trends Theor. Comput. Sci..

[30]  Hyeontaek Lim,et al.  MICA: A Holistic Approach to Fast In-Memory Key-Value Storage , 2014, NSDI.

[31]  Erol Gelenbe,et al.  A Unified Approach to the Evaluation of a Class of Replacement Algorithms , 1973, IEEE Transactions on Computers.

[32]  Mithuna Thottethodi,et al.  Understanding and mitigating the impact of load imbalance in the memory caching tier , 2013, SoCC.

[33]  Michele Garetto,et al.  A unified approach to the performance analysis of caching systems , 2013, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[34]  Keith W. Ross,et al.  Hash routing for collections of shared Web caches , 1997, IEEE Netw..

[35]  Yanghee Choi,et al.  CoRC: Coordinated routing and caching for named data networking , 2014, 2014 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS).