Performance Profiling of Cache Systems at Scale

Large scale in-memory object caches such as memcached are widely used to accelerate popular web sites and to reduce the burden on backend databases. Operation and development teams tuning a cache tier would benefit from knowing answers to questions such as “how much total memory should be allocated to the cache tier?” and “what is the minimum cache size for a given hit rate?” We propose a new lightweight online profiler, MIMIR, that hooks into the replacement policy of each cache server and periodically produces histograms of the overall cache hit rate as a function of memory size. It predicts smaller cache sizes with 99% accuracy on average at high performance. In order to predict the hit rate for larger cache sizes than the current allocation, the metadata for some evicted keys must be available. Keeping track of the metadata for all evicted keys is memory expensive and under intensive workloads will fill up the disk space quickly. We propose a new, fast and memory efficient method for storing a specific amount of evicted metadata with automatic flushing using Counting Filters, an extension of Bloom Filters to support removals. This method predicts the hit rate of a larger cache with 95% accuracy on average. Experiments on the profiler within memcached showed that dynamic hit rate histograms are produced with relatively low drop in throughput. Thus our evaluation suggests that online cache profiling can be a practical tool for improving provisioning of large caches. Arangursvoktun a storum flýtiminniskerfum Trausti Saemundsson Mai 2014