A new multi-objective bi-level programming model for energy and locality aware multi-job scheduling in cloud computing

Abstract How to reduce power consumption of data centers has received worldwide attention. By combining the energy-aware data placement policy and locality-aware multi-job scheduling scheme, we propose a new multi-objective bi-level programming model based on MapReduce to improve the energy efficiency of servers. First, the variation of energy consumption with the performance of servers is taken into account; second, data locality can be adjusted dynamically according to current network state; last but not least, considering that task-scheduling strategies depend directly on data placement policies, we formulate the problem as an integer bi-level programming model. In order to solve the model efficiently, specific-design encoding and decoding methods are introduced. Based on these, a new effective multi-objective genetic algorithm based on MOEA/D is proposed. As there are usually tens of thousands of tasks to be scheduled in the cloud, this is a large-scale optimization problem and a local search operator is designed to accelerate convergent speed of the proposed algorithm. Finally, numerical experiments indicate the effectiveness of the proposed model and algorithm.

[1]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[2]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[3]  Heinrich von Stackelberg,et al.  Stackelberg (Heinrich von) - The Theory of the Market Economy, translated from the German and with an introduction by Alan T. PEACOCK. , 1953 .

[4]  Giang Son Tran,et al.  Two levels autonomic resource management in virtualized IaaS , 2013, Future Gener. Comput. Syst..

[5]  Hong Liu,et al.  Energy proportional datacenter networks , 2010, ISCA.

[6]  J. Bard.  Some properties of the bilevel programming problem , 1991 .

[7]  David G. Stork,et al.  Pattern Classification , 1973 .

[8]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[9]  Yuping Wang,et al.  Energy-Efficient Multi-Job Scheduling Model for Cloud Computing and Its Genetic Algorithm , 2012 .

[10]  David Corne,et al.  The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimisation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[11]  Jeffrey Horn,et al.  The Niched Pareto Genetic Algorithm 2 Applied to the Design of Groundwater Remediation Systems , 2001, EMO.

[12]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[13]  Zhen Xiao,et al.  Dynamic Resource Allocation Using Virtual Machines for Cloud Computing Environment , 2013, IEEE Transactions on Parallel and Distributed Systems.

[14]  Tom White,et al.  Hadoop: The Definitive Guide , 2009 .

[15]  Rajkumar Buyya,et al.  Energy-aware resource allocation heuristics for efficient management of data centers for Cloud computing , 2012, Future Gener. Comput. Syst..

[16]  Feng Zhao,et al.  Energy aware consolidation for cloud computing , 2008, CLUSTER 2008.

[17]  Robert G. Jeroslow,et al.  The polynomial hierarchy and a simple model for competitive analysis , 1985, Math. Program..

[18]  Günter Rudolph,et al.  Finite Markov Chain Results in Evolutionary Computation: A Tour d'Horizon , 1998, Fundam. Informaticae.

[19]  Yuping Wang,et al.  Energy-efficient Task Scheduling Model based on MapReduce for Cloud Computing using Genetic Algorithm , 2012, J. Comput..

[20]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[21]  Luiz André Barroso,et al.  The Case for Energy-Proportional Computing , 2007, Computer.

[22]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[23]  Lixia Han,et al.  A new encoding based genetic algorithm for the traveling salesman problem , 2006 .

[24]  El-Ghazali Talbi,et al.  A multi-start local search heuristic for an energy efficient VMs assignment on top of the OpenNebula cloud manager , 2014, Future Gener. Comput. Syst..

[25]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[26]  P. Mell,et al.  SP 800-145. The NIST Definition of Cloud Computing , 2011 .

[27]  Yuping Wang,et al.  An evolutionary algorithm for solving nonlinear bilevel programming based on a new constraint-handling scheme , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[28]  Lakshmi Ganesh,et al.  Integrated Approach to Data Center Power Management , 2013, IEEE Transactions on Computers.

[29]  Rajkumar Buyya,et al.  Author's Personal Copy Future Generation Computer Systems a Coordinator for Scaling Elastic Applications across Multiple Clouds , 2022 .

[30]  Jinli Cao,et al.  Scheduling para-virtualized virtual machines based on events , 2013, Future Gener. Comput. Syst..

[31]  Adam Wierman,et al.  Data center demand response: avoiding the coincident peak via workload shifting and local generation , 2013, SIGMETRICS '13.