A new energy-aware task scheduling method for data-intensive applications in the cloud

Maximizing energy efficiency while ensuring the user's Service-Level Agreement (SLA) is very important for the purpose of environmental protection and profit maximization for the cloud service providers. In this paper, an energy and deadline aware task scheduling method for data-intensive applications is proposed. In this method, first, the datasets and tasks are modeled as a binary tree by a data correlation clustering algorithm, in which both the data correlations generated from the initial datasets and that from the intermediate datasets have been considered. Hence, the amount of global data transmission can be reduced greatly, which are beneficial to the reduction of SLA violation rate. Second, a "Tree-to-Tree" task scheduling approach based on the calculation of Task Requirement Degree (TRD) is proposed, which can improve energy efficiency of the whole cloud system by reducing the number of active machines, decreasing the global time consumption on data transmission, and optimizing the utilization of its computing resources and network bandwidth. Experiment results show that the power consumption of the cloud system can be reduced efficiently while maintaining a low-level SLA violation rate. A new data correlation clustering method is proposed to reduce data transmission.A "Tree-to-Tree" heuristic energy-aware task scheduling strategy is proposed.The utilization of network bandwidth and computing capacities can be improved.

[1]  Yasuhiro Ajiro,et al.  Improving Packing Algorithms for Server Consolidation , 2007, Int. CMG Conference.

[2]  Chen Yi,et al.  A Data Placement Strategy Based on Genetic Algorithm for Scientific Workflows , 2012, 2012 Eighth International Conference on Computational Intelligence and Security.

[3]  Bingsheng He,et al.  On the Efficiency and Programmability of Large Graph Processing in the Cloud , 2010 .

[4]  Arun Venkataramani,et al.  Sandpiper: Black-box and gray-box resource management for virtual machines , 2009, Comput. Networks.

[5]  Walter Binder,et al.  Opportunistic Service Provisioning in the Cloud , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[6]  Yue Gao,et al.  Using explicit output comparisons for fault tolerant scheduling (FTS) on modern high-performance processors , 2013, 2013 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[7]  Anna Gorbenko,et al.  Task-resource scheduling problem , 2012, International Journal of Automation and Computing.

[8]  Andrzej Kochut,et al.  Dynamic Placement of Virtual Machines for Managing SLA Violations , 2007, 2007 10th IFIP/IEEE International Symposium on Integrated Network Management.

[9]  Alexandru Iosup,et al.  How are Real Grids Used? The Analysis of Four Grid Traces and Its Implications , 2006, 2006 7th IEEE/ACM International Conference on Grid Computing.

[10]  Djamal Zeghlache,et al.  Minimum Cost Maximum Flow Algorithm for Dynamic Resource Allocation in Clouds , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[11]  Maurice Gagnaire,et al.  Resource Provisioning for Enriched Services in Cloud Environment , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[12]  Odej Kao,et al.  Nephele: efficient parallel data processing in the cloud , 2009, MTAGS '09.

[13]  Maurice Gagnaire,et al.  Dynamic Resource Allocation in Cloud Environment Under Time-variant Job Requests , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[14]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[15]  Feng Zhao,et al.  Energy aware consolidation for cloud computing , 2008, CLUSTER 2008.

[16]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[17]  Ishfaq Ahmad,et al.  Benchmarking the task graph scheduling algorithms , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[18]  Suman Nath,et al.  Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services , 2008, NSDI.

[19]  Fatih Alagöz,et al.  A survey of research on greening data centers , 2012, 2012 IEEE Global Communications Conference (GLOBECOM).

[20]  Rajkumar Buyya,et al.  Market-Oriented Cloud Computing: Vision, Hype, and Reality of Delivering Computing as the 5th Utility , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[21]  Zhiliang Zhu,et al.  Dynamic Provisioning Modeling for Virtualized Multi-tier Applications in Cloud Data Center , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[22]  Simsy Xavier A Survey of Various Workflow Scheduling Algorithms in Cloud Environment , 2013 .

[23]  Amit Kumar Das,et al.  Energy-Efficient Scheduling Algorithms for Data Center Resources in Cloud Computing , 2013, 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing.

[24]  Nagarajan Kandasamy,et al.  Power and performance management of virtualized computing environments via lookahead control , 2008, 2008 International Conference on Autonomic Computing.

[25]  Xiao Liu,et al.  A data placement strategy in scientific cloud workflows , 2010, Future Gener. Comput. Syst..

[26]  Yuan Yu,et al.  Dryad: distributed data-parallel programs from sequential building blocks , 2007, EuroSys '07.

[27]  Radu Prodan,et al.  A Multi-objective Approach for Workflow Scheduling in Heterogeneous Environments , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[28]  Meng Wang,et al.  Consolidating virtual machines with dynamic bandwidth demand in data centers , 2011, 2011 Proceedings IEEE INFOCOM.

[29]  Jian Xiao,et al.  A Data Placement Strategy for Data-Intensive Scientific Workflows in Cloud , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[30]  Massoud Pedram,et al.  Energy-Efficient Datacenters , 2012, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[31]  Bruce M. Maggs,et al.  Cutting the electric bill for internet-scale systems , 2009, SIGCOMM '09.

[32]  Dmytro Dyachuk,et al.  Maximizing Cloud Providers' Revenues via Energy Aware Allocation Policies , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[33]  Christof Fetzer,et al.  Energy-aware scheduling for infrastructure clouds , 2012, 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings.