Cost and makespan scheduling of workflows in clouds using list multiobjective optimization technique

Abstract Highly scalable resource supply capacity of cloud computing has greatly improved the execution speed of workflow applications, however, traditional workflow scheduling algorithms which focus on the optimization of makespan (execution time) of workflows, become inappropriate for the design of large-scale workflow systems. Workflow scheduling in cloud computing is particularly a multiobjective optimization problem, in which many critical issues besides the execution time of workflows should be taken into account. Although many heuristics and meta-heuristics have been proposed to solve this problem, most of them cannot produce satisfactory cost-makespan tradeoffs and have a long time overhead. In this paper, we propose an efficient heuristic named CMSWC (Cost and Makespan Scheduling of Workflows in the Cloud) to solve the workflow scheduling problem, by simultaneously minimizing cost and makespan of workflows. CMSCW follows a two-phase list scheduling philosophy: ranking and mapping. Furthermore, CMSCW incorporates with three designs specifically for the multiobjective challenges: (i) The mapping phase is designed to avoid exploring useless resources for tasks, which significantly narrows down the search space. (ii) A new method is proposed to select non-dominated solutions, by combining the quick non-dominated sorting approach and Shift-Based Density Estimation (SDE) based crowding distance. (iii) Several elitist study strategies are designed to make solutions close to the true Pareto front as well as avoid trapping into local optimum. Extensive experiments on real-life workflows demonstrate that our approach can generate better cost-makespan tradeoff fronts than that of several state-of-the-art approaches.

[1]  Fei Xie,et al.  Scheduling non-preemptive tasks with strict periods in multi-core real-time systems , 2018, J. Syst. Archit..

[2]  Radu Prodan,et al.  Multi-objective workflow scheduling in Amazon EC2 , 2014, Cluster Computing.

[3]  Deo Prakash Vidyarthi,et al.  A Cost-Effective Deadline-Constrained Dynamic Scheduling Algorithm for Scientific Workflows in a Cloud Environment , 2018, IEEE Transactions on Cloud Computing.

[4]  Jinjun Chen,et al.  Cost optimization for deadline-aware scheduling of big-data processing jobs on clouds , 2017, Future Gener. Comput. Syst..

[5]  Rajkumar Buyya,et al.  Optimizing the makespan and reliability for workflow applications with reputation and a look-ahead genetic algorithm , 2011, Future Gener. Comput. Syst..

[6]  Miron Livny,et al.  Pegasus, a workflow management system for science automation , 2015, Future Gener. Comput. Syst..

[7]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[8]  Ge Yu,et al.  Minimizing temperature and energy of real-time applications with precedence constraints on heterogeneous MPSoC systems , 2019, J. Syst. Archit..

[9]  Rajesh Devaraj,et al.  Contention-aware optimal scheduling of real-time precedence-constrained task graphs on heterogeneous distributed systems , 2020, J. Syst. Archit..

[10]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Jorge-Arnulfo Quiané-Ruiz,et al.  Runtime measurements in the cloud , 2010, Proc. VLDB Endow..

[12]  Mohammad Masdari,et al.  Towards workflow scheduling in cloud computing: A comprehensive analysis , 2016, J. Netw. Comput. Appl..

[13]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[14]  Junlong Zhou,et al.  Cost and makespan-aware workflow scheduling in hybrid clouds , 2019, J. Syst. Archit..

[15]  Shafii Muhammad Abdulhamid,et al.  An efficient symbiotic organisms search algorithm with chaotic optimization strategy for multi-objective task scheduling problems in cloud computing environment , 2019, J. Netw. Comput. Appl..

[16]  Thomas Fahringer,et al.  GRP-HEFT: A Budget-Constrained Resource Provisioning Scheme for Workflow Scheduling in IaaS Clouds , 2020, IEEE Transactions on Parallel and Distributed Systems.

[17]  Radu Prodan,et al.  MOHEFT: A multi-objective list-based method for workflow scheduling , 2012, 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings.

[18]  Xiaohui Liu,et al.  Evolutionary Multi-Objective Workflow Scheduling in Cloud , 2016, IEEE Transactions on Parallel and Distributed Systems.

[19]  Shengxiang Yang,et al.  Shift-Based Density Estimation for Pareto-Based Algorithms in Many-Objective Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[20]  Dick H. J. Epema,et al.  Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds , 2013, Future Gener. Comput. Syst..

[21]  Kenli Li,et al.  Bi-objective workflow scheduling of the energy consumption and reliability in heterogeneous computing systems , 2017, Inf. Sci..

[22]  Zonghua Gu,et al.  Security-Aware Mapping and Scheduling with Hardware Co-Processors for FlexRay-Based Distributed Embedded Systems , 2016, IEEE Transactions on Parallel and Distributed Systems.

[23]  Abir Chaabani,et al.  An Efficient Chemical Reaction Optimization Algorithm for Multiobjective Optimization , 2015, IEEE Transactions on Cybernetics.

[24]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[25]  Vijayan Sugumaran,et al.  Task scheduling techniques in cloud computing: A literature survey , 2019, Future Gener. Comput. Syst..

[26]  Rajkumar Buyya,et al.  Deadline Based Resource Provisioningand Scheduling Algorithm for Scientific Workflows on Clouds , 2014, IEEE Transactions on Cloud Computing.

[27]  Chase Qishi Wu,et al.  End-to-End Delay Minimization for Scientific Workflows in Clouds under Budget Constraint , 2015, IEEE Transactions on Cloud Computing.

[28]  Savina Bansal,et al.  Energy-cognizant scheduling for preference-oriented fixed-priority real-time tasks , 2020, J. Syst. Archit..

[29]  Jun Zhang,et al.  Multiobjective Cloud Workflow Scheduling: A Multiple Populations Ant Colony System Approach , 2019, IEEE Transactions on Cybernetics.

[30]  Jun Zhang,et al.  An Energy Efficient Ant Colony System for Virtual Machine Placement in Cloud Computing , 2018, IEEE Transactions on Evolutionary Computation.

[31]  Keqin Li,et al.  Fast Functional Safety Verification for Distributed Automotive Applications During Early Design Phase , 2018, IEEE Transactions on Industrial Electronics.

[32]  Sucha Smanchat,et al.  Taxonomies of workflow scheduling problem and techniques in the cloud , 2015, Future Gener. Comput. Syst..

[33]  Haluk Rahmi Topcuoglu,et al.  Neural network based multi-objective evolutionary algorithm for dynamic workflow scheduling in cloud computing , 2020, Future Gener. Comput. Syst..

[34]  Qingsheng Zhu,et al.  Deadline-Constrained Cost Optimization Approaches for Workflow Scheduling in Clouds , 2017, IEEE Transactions on Parallel and Distributed Systems.

[35]  Jian Li,et al.  Cost-efficient task scheduling for executing large programs in the cloud , 2013, Parallel Comput..

[36]  Song Guo,et al.  Task Scheduling for Energy Consumption Constrained Parallel Applications on Heterogeneous Computing Systems , 2020, IEEE Transactions on Parallel and Distributed Systems.

[37]  Bryan Ng,et al.  Budget and Deadline Aware e-Science Workflow Scheduling in Clouds , 2019, IEEE Transactions on Parallel and Distributed Systems.

[38]  Ann L. Chervenak,et al.  Characterizing and profiling scientific workflows , 2013, Future Gener. Comput. Syst..

[39]  Xiao Liu,et al.  Soft error-aware energy-efficient task scheduling for workflow applications in DVFS-enabled cloud , 2018, J. Syst. Archit..

[40]  Zhenyu Liu,et al.  Real-Time scheduling and analysis of parallel tasks on heterogeneous multi-cores , 2020, J. Syst. Archit..

[41]  Rizos Sakellariou,et al.  A Pareto-based approach for CPU provisioning of scientific workflows on clouds , 2019, Future Gener. Comput. Syst..

[42]  Prasanta K. Jana,et al.  A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing , 2018, Future Gener. Comput. Syst..

[43]  Jin Sun,et al.  Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based HEFT , 2019, Future Gener. Comput. Syst..

[44]  Xu Jiang,et al.  Real-time scheduling of parallel tasks with tight deadlines , 2020, J. Syst. Archit..

[45]  Bin Luo,et al.  Cost and Energy Aware Scheduling Algorithm for Scientific Workflows with Deadline Constraint in Clouds , 2018, IEEE Transactions on Services Computing.

[46]  Rajkumar Buyya,et al.  Multiobjective differential evolution for scheduling workflow applications on global Grids , 2009, Concurr. Comput. Pract. Exp..

[47]  Sakshi Kaushal,et al.  A hybrid multi-objective Particle Swarm Optimization for scientific workflow scheduling , 2017, Parallel Comput..