A Hybrid Metaheuristic for Multi-Objective Scientific Workflow Scheduling in a Cloud Environment

Cloud computing has emerged as a high-performance computing environment with a large pool of abstracted, virtualized, flexible, and on-demand resources and services. Scheduling of scientific workflows in a distributed environment is a well-known NP-complete problem and therefore intractable with exact solutions. It becomes even more challenging in the cloud computing platform due to its dynamic and heterogeneous nature. The aim of this study is to optimize multi-objective scheduling of scientific workflows in a cloud computing environment based on the proposed metaheuristic-based algorithm, Hybrid Bio-inspired Metaheuristic for Multi-objective Optimization (HBMMO). The strong global exploration ability of the nature-inspired metaheuristic Symbiotic Organisms Search (SOS) is enhanced by involving an efficient list-scheduling heuristic, Predict Earliest Finish Time (PEFT), in the proposed algorithm to obtain better convergence and diversity of the approximate Pareto front in terms of reduced makespan, minimized cost, and efficient load balance of the Virtual Machines (VMs). The experiments using different scientific workflow applications highlight the effectiveness, practicality, and better performance of the proposed algorithm.

[1]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[2]  Yaochu Jin,et al.  A hybrid instance-intensive workflow scheduling method in private cloud environment , 2017, Natural Computing.

[3]  Hamid Arabnejad,et al.  List Scheduling Algorithm for Heterogeneous Systems by an Optimistic Cost Table , 2014, IEEE Transactions on Parallel and Distributed Systems.

[4]  Poonam Singh,et al.  A review of task scheduling based on meta-heuristics approach in cloud computing , 2017, Knowledge and Information Systems.

[5]  Marc Frîncu,et al.  Scheduling highly available applications on cloud environments , 2014, Future Gener. Comput. Syst..

[6]  Alexandru Iosup,et al.  A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing , 2009, CloudComp.

[7]  Yongsheng Ding,et al.  Endocrine-based coevolutionary multi-swarm for multi-objective workflow scheduling in a cloud system , 2017, Soft Comput..

[8]  Mohammed Abdullahi,et al.  Hybrid Symbiotic Organisms Search Optimization Algorithm for Scheduling of Tasks on Cloud Computing Environment , 2016, PloS one.

[9]  Huifang Deng,et al.  Elastic Scheduling of Scientific Workflows under Deadline Constraints in Cloud Computing Environments , 2018, Future Internet.

[10]  Jemal H. Abawajy,et al.  An efficient meta-heuristic algorithm for grid computing , 2013, Journal of Combinatorial Optimization.

[11]  Xu Zhou,et al.  A Novel Hybrid Multi-Objective Population Migration Algorithm , 2015, Int. J. Pattern Recognit. Artif. Intell..

[12]  Rajkumar Buyya,et al.  A Hybrid Bio-Inspired Algorithm for Scheduling and Resource Management in Cloud Environment , 2020, IEEE Transactions on Services Computing.

[13]  Shulin Tian,et al.  An Adaptive Hybrid PSO Multi-Objective Optimization Algorithm for Constrained Optimization Problems , 2015, Int. J. Pattern Recognit. Artif. Intell..

[14]  Zhen Ji,et al.  A multi-objective memetic algorithm based on locality-sensitive hashing for one-to-many-to-one dynamic pickup-and-delivery problem , 2016, Inf. Sci..

[15]  Albert Y. Zomaya,et al.  A balanced scheduler with data reuse and replication for scientific workflows in cloud computing systems , 2017, Future Gener. Comput. Syst..

[16]  Min-Yuan Cheng,et al.  Symbiotic Organisms Search: A new metaheuristic optimization algorithm , 2014 .

[17]  Xiaohui Liu,et al.  Evolutionary Multi-Objective Workflow Scheduling in Cloud , 2016, IEEE Transactions on Parallel and Distributed Systems.

[18]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[19]  Shafii Muhammad Abdulhamid,et al.  Symbiotic Organism Search optimization based task scheduling in cloud computing environment , 2016, Future Gener. Comput. Syst..

[20]  Ann L. Chervenak,et al.  Characterizing and profiling scientific workflows , 2013, Future Gener. Comput. Syst..

[21]  Felipe Campelo,et al.  Preference-guided evolutionary algorithms for many-objective optimization , 2016, Inf. Sci..

[22]  Arnapurna Panda,et al.  A Symbiotic Organisms Search algorithm with adaptive penalty function to solve multi-objective constrained optimization problems , 2016, Appl. Soft Comput..

[23]  Rajkumar Buyya,et al.  Budget-Driven Scheduling of Scientific Workflows in IaaS Clouds with Fine-Grained Billing Periods , 2017, ACM Trans. Auton. Adapt. Syst..

[24]  Aderemi Oluyinka Adewumi,et al.  Soft sets based symbiotic organisms search algorithm for resource discovery in cloud computing environment , 2017, Future Gener. Comput. Syst..

[25]  Radu Prodan,et al.  Multi-objective workflow scheduling in Amazon EC2 , 2014, Cluster Computing.

[26]  Jorge-Arnulfo Quiané-Ruiz,et al.  Runtime measurements in the cloud , 2010, Proc. VLDB Endow..

[27]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[28]  Prasanta K. Jana,et al.  A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing , 2018, Future Gener. Comput. Syst..

[29]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[30]  Miron Livny,et al.  Pegasus, a workflow management system for science automation , 2015, Future Gener. Comput. Syst..

[31]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[32]  Danny Dolev,et al.  Extensible Architecture for High-Performance, Scalable, Reliable Publish-Subscribe Eventing and Notification , 2007, Int. J. Web Serv. Res..

[33]  Rajkumar Buyya,et al.  Energy-aware resource allocation heuristics for efficient management of data centers for Cloud computing , 2012, Future Gener. Comput. Syst..

[34]  Marty Humphrey,et al.  Auto-scaling to minimize cost and meet application deadlines in cloud workflows , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[35]  Sakshi Kaushal,et al.  A hybrid multi-objective Particle Swarm Optimization for scientific workflow scheduling , 2017, Parallel Comput..

[36]  Ewa Deelman,et al.  WorkflowSim: A toolkit for simulating scientific workflows in distributed environments , 2012, 2012 IEEE 8th International Conference on E-Science.

[37]  Carlos A. Coello Coello,et al.  Evolutionary multi-objective optimization: a historical view of the field , 2006, IEEE Comput. Intell. Mag..

[38]  Keqin Li,et al.  Future Generation Computer Systems ( ) – Future Generation Computer Systems Multi-objective Scheduling of Many Tasks in Cloud Platforms , 2022 .

[39]  Syed Hamid Hussain Madni,et al.  An Appraisal of Meta-Heuristic Resource Allocation Techniques for IaaS Cloud , 2016 .

[40]  Jian Li,et al.  Cost-efficient task scheduling for executing large programs in the cloud , 2013, Parallel Comput..

[41]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[42]  Prasanta K. Jana,et al.  A novel cost-efficient approach for deadline-constrained workflow scheduling by dynamic provisioning of resources , 2018, Future Gener. Comput. Syst..

[43]  Radu Prodan,et al.  Multi-objective list scheduling of workflow applications in distributed computing infrastructures , 2014, J. Parallel Distributed Comput..