Taxonomies of workflow scheduling problem and techniques in the cloud

Scientific workflows, like other applications, benefit from the cloud computing, which offers access to virtually unlimited resources provisioned elastically on demand. In order to efficiently execute a workflow in the cloud, scheduling is required to address many new aspects introduced by cloud resource provisioning. In the last few years, many techniques have been proposed to tackle different cloud environments enabled by the flexible nature of the cloud, leading to the techniques of different designs. In this paper, taxonomies of cloud workflow scheduling problem and techniques are proposed based on analytical review. We identify and explain the aspects and classifications unique to workflow scheduling in the cloud environment in three categories, namely, scheduling process, task and resource. Lastly, review of several scheduling techniques are included and classified onto the proposed taxonomies. We hope that our taxonomies serve as a stepping stone for those entering this research area and for further development of scheduling technique. We propose taxonomies of cloud workflow scheduling problem and techniques.Aspects and classifications unique to cloud workflow scheduling are identified.Several techniques are reviewed and classified based on the proposed taxonomies.Some issues of future concern in cloud workflow scheduling are discussed.

[1]  Francine Berman,et al.  Heuristics for scheduling parameter sweep applications in grid environments , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[2]  Quan Z. Sheng,et al.  Science in the Cloud: Allocation and Execution of Data-Intensive Scientific Workflows , 2013, Journal of Grid Computing.

[3]  Albert Y. Zomaya,et al.  On Resource Efficiency of Workflow Schedules , 2014, ICCS.

[4]  David Abramson,et al.  Scheduling Multiple Parameter Sweep Workflow Instances on the Grid , 2009, 2009 Fifth IEEE International Conference on e-Science.

[5]  Radu Prodan,et al.  A New Fault Tolerance Heuristic for Scientific Workflows in Highly Distributed Environments Based on Resubmission Impact , 2009, 2009 Fifth IEEE International Conference on e-Science.

[6]  Paul Watson.  A multi-level security model for partitioning workflows over federated clouds , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[7]  Amandeep Verma,et al.  Workflow scheduling algorithms in cloud environment - A survey , 2014, 2014 Recent Advances in Engineering and Computational Sciences (RAECS).

[8]  Rajkumar Buyya,et al.  A Particle Swarm Optimization-Based Heuristic for Scheduling Workflow Applications in Cloud Computing Environments , 2010, 2010 24th IEEE International Conference on Advanced Information Networking and Applications.

[9]  Morteza Analoui,et al.  QoS-based scheduling of workflow applications on grids , 2007 .

[10]  Rajkumar Buyya,et al.  Optimizing the makespan and reliability for workflow applications with reputation and a look-ahead genetic algorithm , 2011, Future Gener. Comput. Syst..

[11]  Rajkumar Buyya,et al.  A Dynamic Critical Path Algorithm for Scheduling Scientific Workflow Applications on Global Grids , 2007, Third IEEE International Conference on e-Science and Grid Computing (e-Science 2007).

[12]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[13]  Dick H. J. Epema,et al.  Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds , 2013, Future Gener. Comput. Syst..

[14]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[15]  Sucha Smanchat,et al.  A scheduling algorithm for grid workflow using bottleneck detection and load balancing , 2014, Int. J. Web Inf. Syst..

[16]  Lucio Grandinetti,et al.  An approximate ϵϵ-constraint method for a multi-objective job scheduling in the cloud , 2013, Future Gener. Comput. Syst..

[17]  Enda Barrett,et al.  A Learning Architecture for Scheduling Workflow Applications in the Cloud , 2011, 2011 IEEE Ninth European Conference on Web Services.

[18]  Radu Prodan,et al.  A Multi-objective Approach for Workflow Scheduling in Heterogeneous Environments , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[19]  Radu Prodan,et al.  Impact of Variable Priced Cloud Resources on Scientific Workflow Scheduling , 2012, Euro-Par.

[20]  Jianwu Wang,et al.  Workflow as a Service in the Cloud: Architecture and Scheduling Algorithms , 2014, ICCS.

[21]  Radu Prodan,et al.  Towards a general model of the multi-criteria workflow scheduling on the grid , 2009, Future Gener. Comput. Syst..

[22]  Albert Y. Zomaya,et al.  Rescheduling for reliable job completion with the support of clouds , 2010, Future Gener. Comput. Syst..

[23]  Dick H. J. Epema,et al.  Cost-driven scheduling of grid workflows using Partial Critical Paths , 2010, 2010 11th IEEE/ACM International Conference on Grid Computing.

[24]  Edmundo Roberto Mauro Madeira,et al.  A performance-oriented adaptive scheduler for dependent tasks on grids , 2008 .

[25]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[26]  Rizos Sakellariou,et al.  Budget-Deadline Constrained Workflow Planning for Admission Control , 2013, Journal of Grid Computing.

[27]  Xiao Liu,et al.  A Revised Discrete Particle Swarm Optimization for Cloud Workflow Scheduling , 2010, 2010 International Conference on Computational Intelligence and Security.

[28]  Rajkumar Buyya,et al.  Energy-aware resource allocation heuristics for efficient management of data centers for Cloud computing , 2012, Future Gener. Comput. Syst..

[29]  Rajkumar Buyya,et al.  Workflow scheduling algorithms for grid computing , 2008 .

[30]  Nelson Luis Saldanha da Fonseca,et al.  Scheduling in hybrid clouds , 2012, IEEE Communications Magazine.

[31]  Radu Prodan,et al.  Dynamic Cloud provisioning for scientific Grid workflows , 2010, 2010 11th IEEE/ACM International Conference on Grid Computing.

[32]  Radu Prodan,et al.  Bi-criteria Scheduling of Scientific Workflows for the Grid , 2008, 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID).

[33]  C. Tham,et al.  QoS-based Scheduling of Workflow Applications on Service Grids , 2005 .

[34]  Min Chen,et al.  Cost adaptive workflow scheduling in cloud computing , 2014, ICUIMC '14.

[35]  R. F. Freund,et al.  Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[36]  Ian J. Taylor,et al.  Workflows and e-Science: An overview of workflow system features and capabilities , 2009, Future Gener. Comput. Syst..

[37]  Jack J. Dongarra,et al.  Scheduling workflow applications on processors with different capabilities , 2006, Future Gener. Comput. Syst..

[38]  Marty Humphrey,et al.  Auto-scaling to minimize cost and meet application deadlines in cloud workflows , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[39]  Hai Jin,et al.  A Min-Min average algorithm for scheduling transaction-intensive grid workflows , 2009, AusGrid '09.

[40]  David Abramson,et al.  Electrochemical Parameter Optimization Using Scientific Workflows , 2010, 2010 IEEE Sixth International Conference on e-Science.

[41]  Jun Zhang,et al.  A set-based discrete PSO for cloud workflow scheduling with user-defined QoS constraints , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[42]  Eugenio Zimeo,et al.  Time and Cost-Driven Scheduling of Data Parallel Tasks in Grid Workflows , 2009, IEEE Systems Journal.

[43]  David Abramson,et al.  Scheduling parameter sweep workflow in the Grid based on resource competition , 2013, Future Gener. Comput. Syst..

[44]  Jarek Nabrzyski,et al.  Algorithms for cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds , 2015 .

[45]  Li Liu,et al.  A Survey on Workflow Management and Scheduling in Cloud Computing , 2014, 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[46]  Radu Prodan,et al.  Taxonomies of the Multi-Criteria Grid Workflow Scheduling Problem , 2008 .

[47]  Luiz Fernando Bittencourt,et al.  Workflow scheduling for SaaS / PaaS cloud providers considering two SLA levels , 2012, 2012 IEEE Network Operations and Management Symposium.

[48]  Calvin J. Ribbens,et al.  Hybrid Computing - Where HPC meets grid and Cloud Computing , 2011, Future Gener. Comput. Syst..

[49]  Luiz Fernando Bittencourt,et al.  HCOC: a cost optimization algorithm for workflow scheduling in hybrid clouds , 2011, Journal of Internet Services and Applications.

[50]  Albert Y. Zomaya,et al.  MPHC: Preserving Privacy for Workflow Execution in Hybrid Clouds , 2013, 2013 International Conference on Parallel and Distributed Computing, Applications and Technologies.

[51]  Rajkumar Buyya,et al.  Fault-tolerant Workflow Scheduling using Spot Instances on Clouds , 2014, ICCS.

[52]  Selmin Nurcan,et al.  Bi-criteria Workflow Tasks Allocation and Scheduling in Cloud Computing Environments , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[53]  Wu Wu,et al.  Scheduling Workflow in Cloud Computing Based on Hybrid Particle Swarm Algorithm , 2012 .

[54]  Jun Zhang,et al.  An Ant Colony Optimization Approach to a Grid Workflow Scheduling Problem With Various QoS Requirements , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[55]  Hongbo Liu,et al.  Nature inspired meta-heuristics for grid scheduling: single and multi-objective optimization approaches , 2008 .

[56]  Ke Liu,et al.  Scheduling algorithms for instance-intensive cloud workflows , 2009 .

[57]  Rajkumar Buyya,et al.  InterCloud: Utility-Oriented Federation of Cloud Computing Environments for Scaling of Application Services , 2010, ICA3PP.

[58]  Sucha Smanchat,et al.  Identifying Information Requirement for Scheduling Kepler Workflow in the Cloud , 2014, ICCS.

[59]  DeelmanEwa,et al.  Algorithms for cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds , 2015 .

[60]  Paul Watson,et al.  The case for dynamic security solutions in public cloud workflow deployments , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks Workshops (DSN-W).

[61]  Lúcia Maria de A. Drummond,et al.  Optimizing virtual machine allocation for parallel scientific workflows in federated clouds , 2015, Future Gener. Comput. Syst..

[62]  Sai Peck Lee,et al.  Cost-aware challenges for workflow scheduling approaches in cloud computing environments: Taxonomy and opportunities , 2015, Future Gener. Comput. Syst..

[63]  Radu Prodan,et al.  Multi-objective Workflow Scheduling: An Analysis of the Energy Efficiency and Makespan Tradeoff , 2013, 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing.

[64]  Albert Y. Zomaya,et al.  Resource-efficient workflow scheduling in clouds , 2015, Knowl. Based Syst..

[65]  Jin-Soo Kim,et al.  Estimating Resource Needs for Time-Constrained Workflows , 2008, 2008 IEEE Fourth International Conference on eScience.

[66]  Rajkumar Buyya,et al.  Cost-based scheduling of scientific workflow applications on utility grids , 2005, First International Conference on e-Science and Grid Computing (e-Science'05).

[67]  Roland H. C. Yap,et al.  Tagged-MapReduce: A General Framework for Secure Computing with Mixed-Sensitivity Data on Hybrid Clouds , 2014, 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[68]  Xiaorong Li,et al.  SABA: A security-aware and budget-aware workflow scheduling strategy in clouds , 2015, J. Parallel Distributed Comput..

[69]  Kouichi Sakurai,et al.  Reliable workflow scheduling with less resource redundancy , 2013, Parallel Comput..

[70]  Jarek Nabrzyski,et al.  Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[71]  Sudhir Shenai,et al.  Survey on Scheduling Issues in Cloud Computing , 2012 .

[72]  Jin-Soo Kim,et al.  BTS: Resource capacity estimate for time-targeted science workflows , 2011, J. Parallel Distributed Comput..

[73]  Jin-Soo Kim,et al.  Cost optimized provisioning of elastic resources for application workflows , 2011, Future Gener. Comput. Syst..

[74]  David Abramson,et al.  Nimrod/K: Towards massively parallel dynamic Grid workflows , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[75]  YONGHONG LUO,et al.  Power Consumption Optimization Strategy of Cloud Workflow Scheduling Based on SLA , 2014 .

[76]  Daniel A. Menascé,et al.  A framework for resource allocation in grid computing , 2004, The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, 2004. (MASCOTS 2004). Proceedings..

[77]  Rajkumar Buyya,et al.  Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms , 2006, Sci. Program..

[78]  Dave Cliff,et al.  A financial brokerage model for cloud computing , 2011, Journal of Cloud Computing: Advances, Systems and Applications.

[79]  Dana Petcu,et al.  MODAClouds: A model-driven approach for the design and execution of applications on multiple Clouds , 2012, 2012 4th International Workshop on Modeling in Software Engineering (MISE).

[80]  Xiaorong Li,et al.  ScaleStar: Budget Conscious Scheduling Precedence-Constrained Many-task Workflow Applications in Cloud , 2012, 2012 IEEE 26th International Conference on Advanced Information Networking and Applications.

[81]  Xiao Liu,et al.  A market-oriented hierarchical scheduling strategy in cloud workflow systems , 2011, The Journal of Supercomputing.