Seadown: SLA-Aware Size-Scaling Power Management in Heterogeneous MapReduce Cluster

Power consumption accounts for a large proportion of operating cost in data centers which adds substantially to an organization's power bills and carbon footprint,while much of the energy is wasted.One class of works are seeks to turn off servers for power saving during low utilization period,but most of them are highly constrained by data layout and performance penalty.Arbitrarily powering down servers that are running data-intensive applications is problematic,since it rends data loss,decreases the ability of fault tolerance and affects processing speed.This paper proposes a SLA-aware size-scaling framework named Seadown in heterogeneous MapReduce cluster.The authors first design a hybrid data layout policy which allows turning off large amount of nodes without data lose,and brings high rebuild parallelism in case of failure,then propose a pre-knowledge based workload runtime estimation method which accurately predicts the performance in various cluster configurations.By holding this detailed information,selectively turn nodes off with the purpose of minimizing the energy consumption as well as meeting the performance requirement.The authors prove the NP-hardness of the targeted problem and propose a fine-grained heuristic algorithm to power down servers.Through comprehensive experiments,it is found that the relative errors of the runtime estimation are mostly below 6% and the Seadown framework can effectively cut large portion of energy consumption while meeting performance requirement.