Evolutionary Multi-Objective Workflow Scheduling in Cloud

Cloud computing provides promising platforms for executing large applications with enormous computational resources to offer on demand. In a Cloud model, users are charged based on their usage of resources and the required quality of service (QoS) specifications. Although there are many existing workflow scheduling algorithms in traditional distributed or heterogeneous computing environments, they have difficulties in being directly applied to the Cloud environments since Cloud differs from traditional heterogeneous environments by its service-based resource managing method and pay-per-use pricing strategies. In this paper, we highlight such difficulties, and model the workflow scheduling problem which optimizes both makespan and cost as a Multi-objective Optimization Problem (MOP) for the Cloud environments. We propose an evolutionary multi-objective optimization (EMO)-based algorithm to solve this workflow scheduling problem on an infrastructure as a service (IaaS) platform. Novel schemes for problem-specific encoding and population initialization, fitness evaluation and genetic operators are proposed in this algorithm. Extensive experiments on real world workflows and randomly generated workflows show that the schedules produced by our evolutionary algorithm present more stability on most of the workflows with the instance-based IaaS computing and pricing models. The results also show that our algorithm can achieve significantly better solutions than existing state-of-the-art QoS optimization scheduling algorithms in most cases. The conducted experiments are based on the on-demand instance types of Amazon EC2; however, the proposed algorithm are easy to be extended to the resources and pricing models of other IaaS services.

[1]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[2]  Ritu Garg,et al.  Multi-objective Workflow Grid Scheduling Based on Discrete Particle Swarm Optimization , 2011, SEMCCO.

[3]  Xiaodong Li,et al.  A Non-dominated Sorting Particle Swarm Optimizer for Multiobjective Optimization , 2003, GECCO.

[4]  Hisao Ishibuchi,et al.  Many-Objective Test Problems to Visually Examine the Behavior of Multiobjective Evolution in a Decision Space , 2010, PPSN.

[5]  Ritu Garg,et al.  Multi-objective workflow grid scheduling using $$\varepsilon $$ε-fuzzy dominance sort based discrete particle swarm optimization , 2014, The Journal of Supercomputing.

[6]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[7]  Jian Li,et al.  Cost-efficient task scheduling for executing large programs in the cloud , 2013, Parallel Comput..

[8]  Daniel S. Katz,et al.  Pegasus: A framework for mapping complex scientific workflows onto distributed systems , 2005, Sci. Program..

[9]  Dick H. J. Epema,et al.  Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds , 2013, Future Gener. Comput. Syst..

[10]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[11]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[12]  Mei-Hui Su,et al.  Characterization of scientific workflows , 2008, 2008 Third Workshop on Workflows in Support of Large-Scale Science.

[13]  Jarek Nabrzyski,et al.  Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[14]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[15]  Shengxiang Yang,et al.  ETEA: A Euclidean Minimum Spanning Tree-Based Evolutionary Algorithm for Multi-Objective Optimization , 2014, Evolutionary Computation.

[16]  G. Bruce Berriman,et al.  Comparing FutureGrid, Amazon EC2, and Open Science Grid for Scientific Workflows , 2013, Computing in Science & Engineering.

[17]  Qingfu Zhang,et al.  Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II , 2009, IEEE Transactions on Evolutionary Computation.

[18]  Shengxiang Yang,et al.  Evolutionary Algorithms With Segment-Based Search for Multiobjective Optimization Problems , 2014, IEEE Transactions on Cybernetics.

[19]  Rizos Sakellariou,et al.  Budget-Deadline Constrained Workflow Planning for Admission Control , 2013, Journal of Grid Computing.

[20]  Hamid Arabnejad,et al.  A Budget Constrained Scheduling Algorithm for Workflow Applications , 2014, Journal of Grid Computing.

[21]  Frank Teuteberg,et al.  Costing of Cloud Computing Services: A Total Cost of Ownership Approach , 2012, 2012 45th Hawaii International Conference on System Sciences.

[22]  Rajkumar Buyya,et al.  Multiobjective differential evolution for scheduling workflow applications on global Grids , 2009, Concurr. Comput. Pract. Exp..

[23]  Jun Qin,et al.  ASKALON: a Grid application development and computing environment , 2005, The 6th IEEE/ACM International Workshop on Grid Computing, 2005..

[24]  Cheng Wu,et al.  Ordinal Optimized Scheduling of Scientific Workflows in Elastic Compute Clouds , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[25]  Jun Zhang,et al.  An Ant Colony Optimization Approach to a Grid Workflow Scheduling Problem With Various QoS Requirements , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[26]  Radu Prodan,et al.  Multi-objective list scheduling of workflow applications in distributed computing infrastructures , 2014, J. Parallel Distributed Comput..

[27]  Chase Qishi Wu,et al.  A cost-effective scheduling algorithm for scientific workflows in clouds , 2012, 2012 IEEE 31st International Performance Computing and Communications Conference (IPCCC).

[28]  Radu Prodan,et al.  Multi-objective workflow scheduling in Amazon EC2 , 2014, Cluster Computing.

[29]  Shengxiang Yang,et al.  Shift-Based Density Estimation for Pareto-Based Algorithms in Many-Objective Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[30]  Eui-Nam Huh,et al.  Cost and efficiency-based scheduling on a general framework combining between cloud computing and local thick clients , 2013, 2013 International Conference on Computing, Management and Telecommunications (ComManTel).

[31]  Marios D. Dikaiakos,et al.  Scheduling Workflows with Budget Constraints , 2007, Grid 2007.

[32]  T. Achalakul,et al.  A multiple-objective workflow scheduling framework for cloud data analytics , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).

[33]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[34]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[35]  Rajkumar Buyya,et al.  Multi-objective planning for workflow execution on Grids , 2007, 2007 8th IEEE/ACM International Conference on Grid Computing.

[36]  Radu Prodan,et al.  Towards a general model of the multi-criteria workflow scheduling on the grid , 2009, Future Gener. Comput. Syst..

[37]  Ann L. Chervenak,et al.  Characterizing and profiling scientific workflows , 2013, Future Gener. Comput. Syst..