Electro: Toward QoS-Aware Power Management for Latency-Critical Applications

Reducing the energy consumption of datacenters is critical for their scalability, sustainability, and affordability when hosting latency-critical applications. Prior work has focused on single-thread applications with a stable workload. Recently, multi-thread latency-sensitive services are widely used in current datacenters. However, the variability of user queries in these service makes existing schemes ineffective, leading to either QoS violations or higher energy consumption. In order to address this new problem, we propose Electro, a machine learning enhanced dynamic power management system. Electro consists of a query duration predictor and a query consolidating engine. The duration predictor can precisely predict the duration of each user query in different scenarios based on the pre- trained duration models. At runtime, according to the predicted duration, the query consolidating engine consolidates user queries accordingly to maximize the duration of the CPU idle states while guaranteeing the QoS. The longer each idle state is, the deeper low-power sleep states can the CPU enter. Our evaluation results on the latest Intel Xeon V4 CPU show that Electro reduces the energy consumption by 81.8% on average compared with the default OS scheduling, and by 14.4% on average compared with the state-of-the-art technique while achieving the 95%-ile latency target for latency-sensitive applications.