Multi-objective scheduling for scientific workflow in multicloud environment

Abstract Providing resources and services from multiple clouds is becoming an increasingly promising paradigm. Workflow applications are becoming increasingly computation-intensive or data-intensive, with its resource requirement being maintained from multicloud environment in terms of pay-per-use pricing mechanism. Existing works of cloud workflow scheduling primarily target optimizing makespan or cost. However, the reliability of workflow scheduling is also a critical concern and even the most important metric of QoS (quality of service). In this paper, a multi-objective scheduling (MOS) algorithm for scientific workflow in multicloud environment is proposed, the aim of which is to minimize workflow makespan and cost simultaneously while satisfying the reliability constraint. The proposed MOS algorithm is according to particle swarm optimization (PSO) technology, and the corresponding coding strategy takes both the tasks execution location and tasks order of data transmission into consideration. On the basis of real-world scientific workflow models, extensive simulation experiments demonstrate the significant multi-objective performances improvement of MOS algorithm over the CMOHEFT algorithm and the RANDOM algorithm.

[1]  Muhammad Khurram Khan,et al.  Information collection centric techniques for cloud resource management: Taxonomy, analysis and challenges , 2017, J. Netw. Comput. Appl..

[2]  Salman Mohagheghi,et al.  Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems , 2008, IEEE Transactions on Evolutionary Computation.

[3]  Shengxiang Yang,et al.  Shift-Based Density Estimation for Pareto-Based Algorithms in Many-Objective Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[4]  Hans-Arno Jacobsen,et al.  Scientific Workflow Mining in Clouds , 2017, IEEE Transactions on Parallel and Distributed Systems.

[5]  Saeid Abrishami,et al.  Scheduling Data-Driven Workflows in Multi-cloud Environment , 2015, 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom).

[6]  Rajkumar Buyya,et al.  Multi-objective planning for workflow execution on Grids , 2007, 2007 8th IEEE/ACM International Conference on Grid Computing.

[7]  Rajkumar Buyya,et al.  Meeting Deadlines of Scientific Workflows in Public Clouds with Tasks Replication , 2014, IEEE Transactions on Parallel and Distributed Systems.

[8]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[9]  Soonwook Hwang,et al.  Grid workflow: a flexible failure handling framework for the grid , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[10]  Gang Sun,et al.  Live Migration for Multiple Correlated Virtual Machines in Cloud-Based Data Centers , 2018, IEEE Transactions on Services Computing.

[11]  Albert Y. Zomaya,et al.  An integrated task computation and data management scheduling strategy for workflow applications in cloud environments , 2015, J. Netw. Comput. Appl..

[12]  Naixue Xiong,et al.  Cost-Driven Scheduling for Deadline-Constrained Workflow on Multi-clouds , 2015, 2015 IEEE International Parallel and Distributed Processing Symposium Workshop.

[13]  Cees T. A. M. de Laat,et al.  Defining Intercloud Security Framework and Architecture Components for Multi-cloud Data Intensive Applications , 2017, 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[14]  Radu Prodan,et al.  A Multi-objective Approach for Workflow Scheduling in Heterogeneous Environments , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[15]  LiGuo Huang,et al.  A security and cost aware scheduling algorithm for heterogeneous tasks of scientific workflow in clouds , 2016, Future Gener. Comput. Syst..

[16]  K. Deb An Efficient Constraint Handling Method for Genetic Algorithms , 2000 .

[17]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[18]  Shafii Muhammad Abdulhamid,et al.  Resource scheduling for infrastructure as a service (IaaS) in cloud computing: Challenges and opportunities , 2016, J. Netw. Comput. Appl..

[19]  Neeraj Suri,et al.  SLA-Based Service Selection for Multi-Cloud Environments , 2017, 2017 IEEE International Conference on Edge Computing (EDGE).

[20]  Rajkumar Buyya,et al.  An Algorithm for Network and Data-aware Placement of Multi-Tier Applications in Cloud Data Centers , 2017, J. Netw. Comput. Appl..

[21]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[22]  Mehdi Kargahi,et al.  Reliability-driven scheduling of time/cost-constrained grid workflows , 2016, Future Gener. Comput. Syst..

[23]  Xiaohui Liu,et al.  Evolutionary Multi-Objective Workflow Scheduling in Cloud , 2016, IEEE Transactions on Parallel and Distributed Systems.

[24]  Yuping Wang,et al.  Improving Multiobjective Evolutionary Algorithm by Adaptive Fitness and Space Division , 2005, ICNC.

[25]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[26]  Jonathan E. Fieldsend,et al.  A MOPSO Algorithm Based Exclusively on Pareto Dominance Concepts , 2005, EMO.

[27]  Kenli Li,et al.  Bi-objective workflow scheduling of the energy consumption and reliability in heterogeneous computing systems , 2017, Inf. Sci..

[28]  Kakali Chatterjee,et al.  Cloud security issues and challenges: A survey , 2017, J. Netw. Comput. Appl..

[29]  Rajkumar Buyya,et al.  Location-aware brokering for consumers in multi-cloud computing environments , 2017, J. Netw. Comput. Appl..

[30]  T. Achalakul,et al.  A multiple-objective workflow scheduling framework for cloud data analytics , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).

[31]  Emmanuel Jeannot,et al.  Optimizing performance and reliability on heterogeneous parallel systems: Approximation algorithms and heuristics , 2012, J. Parallel Distributed Comput..

[32]  Xuan Wang,et al.  Resource provision algorithms in cloud computing: A survey , 2016, J. Netw. Comput. Appl..

[33]  Mengjie Zhang,et al.  A memetic particle swarm optimization for constrained multi-objective optimization problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[34]  Ann L. Chervenak,et al.  Characterizing and profiling scientific workflows , 2013, Future Gener. Comput. Syst..

[35]  Prasanta K. Jana,et al.  Compute-intensive workflow scheduling in multi-cloud environment , 2016, 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[36]  Xiaorong Li,et al.  SABA: A security-aware and budget-aware workflow scheduling strategy in clouds , 2015, J. Parallel Distributed Comput..

[37]  Rajkumar Buyya,et al.  Multiobjective differential evolution for scheduling workflow applications on global Grids , 2009 .

[38]  Bin Luo,et al.  Cost and Energy Aware Scheduling Algorithm for Scientific Workflows with Deadline Constraint in Clouds , 2018, IEEE Transactions on Services Computing.

[39]  Yudi Wei,et al.  QoS Guarantees and Service Differentiation for Dynamic Cloud Applications , 2013, IEEE Transactions on Network and Service Management.

[40]  Radu Prodan,et al.  MOHEFT: A multi-objective list-based method for workflow scheduling , 2012, 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings.

[41]  Gary G. Yen,et al.  PSO-Based Multiobjective Optimization With Dynamic Population Size and Adaptive Local Archives , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[42]  Victor I. Chang,et al.  A model to compare cloud and non-cloud storage of Big Data , 2016, Future Gener. Comput. Syst..

[43]  Radu Prodan,et al.  Multi-objective list scheduling of workflow applications in distributed computing infrastructures , 2014, J. Parallel Distributed Comput..

[44]  Yahya Slimani,et al.  A survey on cloud service description , 2017, J. Netw. Comput. Appl..

[45]  Rajkumar Buyya,et al.  Enhancing Reliability of Workflow Execution Using Task Replication and Spot Instances , 2016, ACM Trans. Auton. Adapt. Syst..

[46]  Qingfu Zhang,et al.  Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II , 2009, IEEE Transactions on Evolutionary Computation.