Distributed Load Balancing in Key-Value Networked Caches

Modern web services rely on a network of distributed cache servers to efficiently deliver content to users. Load imbalance among cache servers can substantially degrade content delivery performance. Due to the skewed and dynamic nature of real-world workloads, cache servers that serve viral content experience higher load as compared to other cache servers. We propose a novel distributed load balancing protocol called Meezan to address the load imbalance among cache servers. Meezan replicates popular objects to mitigate skewness and adjusts hash space boundaries in response to load dynamics in a novel way. Our theoretical analysis shows that Meezan achieves near perfect load balancing for a wide range of operating parameters. Our trace driven simulations shows that Meezan reduces load imbalance by up to 52% as compared to prior solutions.

[1]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[2]  Mithuna Thottethodi,et al.  Understanding and mitigating the impact of load imbalance in the memory caching tier , 2013, SoCC.

[3]  Jia Wang,et al.  A survey of web caching schemes for the Internet , 1999, CCRV.

[4]  Prashant J. Shenoy,et al.  Blink: managing server clusters on intermittent power , 2011, ASPLOS XVI.

[5]  Ali Raza Butt,et al.  An in-memory object caching framework with adaptive load balancing , 2015, EuroSys.

[6]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[7]  Sachin Katti,et al.  Dynacache: Dynamic Cloud Caching , 2015, HotStorage.

[8]  Babak Falsafi,et al.  The Case for RackOut: Scalable Data Serving Using Rack-Scale Systems , 2016, SoCC.

[9]  Robbert van Renesse,et al.  Characterizing Load Imbalance in Real-World Networked Caches , 2014, HotNets.

[10]  Muhammad Zubair Shafiq,et al.  Revisiting caching in content delivery networks , 2014, SIGMETRICS '14.

[11]  David R. Karger,et al.  Web Caching with Consistent Hashing , 1999, Comput. Networks.

[12]  Hui Ding,et al.  TAO: Facebook's Distributed Data Store for the Social Graph , 2013, USENIX Annual Technical Conference.

[13]  Bettina Kemme,et al.  AdaptCache: Adaptive Data Partitioning and Migration for Distributed Object Caches , 2016, Middleware.

[14]  Fan Yang,et al.  Proteus: Power Proportional Memory Cache Cluster in Data Centers , 2013, 2013 IEEE 33rd International Conference on Distributed Computing Systems.

[15]  Liqiong Liu,et al.  CircularCache: Scalable and Adaptive Cache Management for Massive Storage Systems , 2016, 2016 IEEE International Conference on Networking, Architecture and Storage (NAS).

[16]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[17]  Muhammad Zubair Shafiq,et al.  Characterizing caching workload of a large commercial Content Delivery Network , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[18]  Tony Tung,et al.  Scaling Memcache at Facebook , 2013, NSDI.

[19]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[20]  G. Barish,et al.  World Wide Web caching: trends and techniques , 2000, IEEE Commun. Mag..

[21]  David R. Karger,et al.  Diminished Chord: A Protocol for Heterogeneous Subgroup Formation in Peer-to-Peer Networks , 2004, IPTPS.

[22]  Yingwei Luo,et al.  LAMA: Optimized Locality-aware Memory Allocation for Key-value Cache , 2015, USENIX Annual Technical Conference.

[23]  Timothy Wood,et al.  Adaptive Performance-Aware Distributed Memory Caching , 2013, ICAC.

[24]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[25]  Rajkumar Buyya,et al.  Dynamically scaling applications in the cloud , 2011, CCRV.

[26]  Song Jiang,et al.  Workload analysis of a large-scale key-value store , 2012, SIGMETRICS '12.

[27]  Van Jacobson,et al.  Adaptive web caching: towards a new global caching architecture , 1998, Comput. Networks.