Balancing Fairness and Efficiency for Cache Sharing in Semi-external Memory System

Data caching and sharing is an effective approach for achieving high performance to many applications in shared platforms such as the cloud. DRAM and SSD are two popular caching devices widely used by many large-scale data application systems such Hadoop and Spark. Due to the limited size of DRAM as well as the large access latency of SSD (relative to DRAM), there is a trend of integrating DRAM and SSD (called semi-external memory) together for large-scale data caching. In this paper, we focus on the semi-external memory cache sharing for multiple users/applications. Two critical metrics, i.e., fairness and efficiency, are considered for the semi-external memory with several challenges. First, it should be sensitive to DRAM and SSD for the semi-external memory in view of their different access latencies. Second, it is crucial to have a policy that can balance fairness and efficiency elastically since there tends to be a tradeoff between them. Third, there is a cheating problem for efficiency cache allocation and we should have a robust allocation policy to address it. We propose a new policy called ElasticSEM for the semi-external memory. It performs the fair allocation of cache resources as a whole with the awareness of different access latencies between DRAM and SSD. Moreover, it contains a knob that allows users to tune and balance fairness and performance flexibly with a guarantee of θ-relaxed fairness, where θ-relaxed fairness refers to as the maximum difference of estimated cache resource allocations between any two users in SEM. Finally, we implement ElasticSEM in an in-memory storage system called Alluxio. The testbed experimental results show that ElasticSEM can achieve high performance and fairness.