Heterogeneity-Aware Workload Placement and Migration in Distributed Sustainable Datacenters

While major cloud service operators have taken various initiatives to operate their sustainable data enters with green energy, it is challenging to effectively utilize the green energy since its generation depends on dynamic natural conditions. Fortunately, the geographical distribution of data enters provides an opportunity for optimizing the system performance by distributing cloud workloads. In this paper, we propose a holistic heterogeneity-aware cloud workload placement and migration approach, sCloud, that aims to maximize the system good put in distributed self-sustainable data enters. sCloud adaptively places the transactional workload to distributed data enters, allocates the available resource to heterogeneous workloads in each data enter, and migrates batch jobs across data enters, while taking into account the green power availability and QoS requirements. We formulate the transactional workload placement as a constrained optimization problem that can be solved by nonlinear programming. Then, we propose a batch job migration algorithm to further improve the system good put when the green power supply varies widely at different locations. We have implemented sCloud in a university cloud test bed with real-world weather conditions and workload traces. Experimental results demonstrate sCloud can achieve near-to-optimal system performance while being resilient to dynamic power availability. It outperforms a heterogeneity-oblivious approach by 26% in improving system good put and 29% in reducing QoS violations.

[1]  Jerome A. Rolia,et al.  Capacity planning and power management to exploit sustainable energy , 2010, 2010 International Conference on Network and Service Management.

[2]  Roy H. Campbell,et al.  Resource Provisioning Framework for MapReduce Jobs with Performance Goals , 2011, Middleware.

[3]  Prashant J. Shenoy,et al.  Cloudy Computing: Leveraging Weather Forecasts in Energy Harvesting Sensor Systems , 2010, 2010 7th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON).

[4]  Austin Donnelly,et al.  Sierra: practical power-proportionality for data center storage , 2011, EuroSys '11.

[5]  Tao Li,et al.  SolarTune: Real-time scheduling with load tuning for solar energy powered multicore systems , 2013, 2013 IEEE 19th International Conference on Embedded and Real-Time Computing Systems and Applications.

[6]  Archana Ganapathi,et al.  The Case for Evaluating MapReduce Performance Using Workload Suites , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[7]  Kang G. Shin,et al.  Automated control of multiple virtualized resources , 2009, EuroSys '09.

[8]  Ming Zhao,et al.  Profit Aware Load Balancing for Distributed Cloud Data Centers , 2013, 2013 IEEE 27th International Symposium on Parallel and Distributed Processing.

[9]  Guillaume Pierre,et al.  Wikipedia workload analysis for decentralized hosting , 2009, Comput. Networks.

[10]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[11]  Lachlan L. H. Andrew,et al.  Geographical load balancing with renewables , 2011, PERV.

[12]  Chao Li,et al.  Enabling distributed generation powered sustainable high-performance data center , 2013, 2013 IEEE 19th International Symposium on High Performance Computer Architecture (HPCA).

[13]  Xue Liu,et al.  Minimizing Electricity Cost: Optimization of Distributed Internet Data Centers in a Multi-Electricity-Market Environment , 2010, 2010 Proceedings IEEE INFOCOM.

[14]  Christopher Stewart,et al.  Adaptive green hosting , 2012, ICAC '12.

[15]  Xiaobo Zhou,et al.  NINEPIN: Non-invasive and energy efficient performance isolation in virtualized servers , 2012, IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012).

[16]  Tajana Rosing,et al.  Utilizing green energy prediction to schedule mixed batch and service jobs in data centers , 2011, OPSR.

[17]  Xiaobo Zhou,et al.  Automated and Agile Server Parameter Tuning with Learning and Control , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium.

[18]  Jordi Torres,et al.  GreenHadoop: leveraging green energy in data-processing frameworks , 2012, EuroSys '12.

[19]  Anand Sivasubramaniam,et al.  Energy storage in datacenters: what, where, and how much? , 2012, SIGMETRICS '12.

[20]  Yefu Wang,et al.  GreenWare: Greening Cloud-Scale Data Centers to Maximize the Use of Renewable Energy , 2011, Middleware.

[21]  Wu-chun Feng,et al.  pVOCL: Power-Aware Dynamic Placement and Migration in Virtualized GPU Environments , 2013, 2013 IEEE 33rd International Conference on Distributed Computing Systems.

[22]  Xiaobo Zhou,et al.  Autonomic performance and power control for co-located Web applications on virtualized servers , 2013, 2013 IEEE/ACM 21st International Symposium on Quality of Service (IWQoS).

[23]  Adam Wierman,et al.  Renewable and cooling aware workload management for sustainable data centers , 2012, SIGMETRICS '12.

[24]  Matei Zaharia,et al.  Job Scheduling for Multi-User MapReduce Clusters , 2009 .

[25]  Yanpei Chen,et al.  Energy efficiency for large-scale MapReduce workloads with significant interactive analysis , 2012, EuroSys '12.

[26]  Prashant J. Shenoy,et al.  Yank: Enabling Green Data Centers to Pull the Plug , 2013, NSDI.

[27]  Xiaobo Zhou,et al.  Self-Tuning Batching with DVFS for Improving Performance and Energy Efficiency in Servers , 2013, 2013 IEEE 21st International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems.

[28]  Xiaoqiao Meng,et al.  Delay tails in MapReduce scheduling , 2012, SIGMETRICS '12.

[29]  Shaolei Ren,et al.  COCA: Online distributed resource management for cost minimization and carbon neutrality in data centers , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[30]  Jordi Torres,et al.  Autonomic Placement of Mixed Batch and Transactional Workloads , 2012, IEEE Transactions on Parallel and Distributed Systems.