Profiling the scheduling decisions for handling critical paths in deadline-constrained cloud workflows

Abstract In this paper, we study the scheduling decisions for handling deadline-constrained workflows in the context of planning customized virtual infrastructures in the cloud. We specifically focus on the effects of using different types of greediness in selecting cost-effective virtual machines for the tasks in an application’s workflow graph. The profiling procedure followed demonstrates that for the widely used approach of the partial critical path algorithm a greedy version is preferred to a more stringent version under different stress conditions, from tight to loose deadlines. Representative topologies of workflow applications are used to generate sets of task graph scheduling problems. Monitoring the performance of the partial critical path algorithm with different types of greediness reveals which of the topologies tested are difficult to solve under various stress conditions. It turns out that an invalid outcome of a greedy version of the partial critical path algorithm is more susceptible to become valid via a final refinement cycle than a less greedy version. The procedure outlined in this paper will allow for a systematic study of a specific heuristic in a workflow scheduling method to increase its success in infrastructure planning under different deadline conditions and is proposed to be part of a general profiling framework.

[1]  Y.-K. Kwok,et al.  Static scheduling algorithms for allocating directed task graphs to multiprocessors , 1999, CSUR.

[2]  Cees T. A. M. de Laat,et al.  Planning virtual infrastructures for time critical applications with multiple deadline constraints , 2017, Future Gener. Comput. Syst..

[3]  Sucha Smanchat,et al.  Taxonomies of workflow scheduling problem and techniques in the cloud , 2015, Future Gener. Comput. Syst..

[4]  Cees T. A. M. de Laat,et al.  ECSched: Efficient Container Scheduling on Heterogeneous Clusters , 2018, Euro-Par.

[5]  Kok Lay Teo,et al.  A review of methods and algorithms for optimizing construction scheduling , 2013, J. Oper. Res. Soc..

[6]  Qingbo Wu,et al.  Workflow scheduling in cloud: a survey , 2015, The Journal of Supercomputing.

[7]  Lida Xu,et al.  An integrated information system for snowmelt flood early-warning based on internet of things , 2013, Information Systems Frontiers.

[8]  Xiaohui Liu,et al.  Evolutionary Multi-Objective Workflow Scheduling in Cloud , 2016, IEEE Transactions on Parallel and Distributed Systems.

[9]  James E. Kelley,et al.  Critical-Path Planning and Scheduling: Mathematical Basis , 1961 .

[10]  Cees T. A. M. de Laat,et al.  Empowering Dynamic Task-Based Applications with Agile Virtual Infrastructure Programmability , 2018, 2018 IEEE 11th International Conference on Cloud Computing (CLOUD).

[11]  Yi-Zeng Liang,et al.  The Matrix Expression, Topological Index and Atomic Attribute of Molecular Topological Structure , 2021, Journal of Data Science.

[12]  Nassim Nicholas Taleb,et al.  The Black Swan: The Impact of the Highly Improbable , 2007 .

[13]  Jamie Kettleborough,et al.  High-resolution global climate modelling: the UPSCALE project, a large-simulation campaign , 2014 .

[14]  Dick H. J. Epema,et al.  Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds , 2013, Future Gener. Comput. Syst..

[15]  James E. Kelley,et al.  Critical-path planning and scheduling , 1899, IRE-AIEE-ACM '59 (Eastern).

[16]  Rizos Sakellariou,et al.  Budget-Deadline Constrained Workflow Planning for Admission Control , 2013, Journal of Grid Computing.

[17]  Li Zhao,et al.  SCEC CyberShake Workflows - Automating Probabilistic Seismic Hazard Analysis Calculations , 2007, Workflows for e-Science, Scientific Workflows for Grids.

[18]  Dimosthenis Kyriazis,et al.  Challenges Emerging from Future Cloud Application Scenarios , 2015, Cloud Forward.

[19]  Sharon A. Curtis,et al.  The classification of greedy algorithms , 2003, Sci. Comput. Program..

[20]  Cheng Wang,et al.  A Survey of Job Scheduling in Grids , 2007, APWeb/WAIM.

[21]  Jean-Marc Vincent,et al.  Random graph generation for scheduling simulations , 2010, SimuTools.

[22]  Dick H. J. Epema,et al.  The Impact of Task Runtime Estimate Accuracy on Scheduling Workloads of Workflows , 2018, 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[23]  Rajiv Ranjan,et al.  Osmotic Flow: Osmotic Computing + IoT Workflow , 2017, IEEE Cloud Computing.

[24]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[25]  Bryan N. Lawrence,et al.  High-resolution global climate modelling: the UPSCALE project, a large-simulation campaign , 2014 .

[26]  Xiaolong Xu,et al.  An ACO-LB Algorithm for Task Scheduling in the Cloud Environment , 2014, J. Softw..

[27]  Fairouz Fakhfakh,et al.  Workflow Scheduling in Cloud Computing: A Survey , 2014, 2014 IEEE 18th International Enterprise Distributed Object Computing Conference Workshops and Demonstrations.

[28]  M. A. Cleveland,et al.  The Problem With Critical Path Scheduling Algorithms , 1996 .

[29]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[30]  Hamid Arabnejad,et al.  List Scheduling Algorithm for Heterogeneous Systems by an Optimistic Cost Table , 2014, IEEE Transactions on Parallel and Distributed Systems.

[31]  Tao Yang,et al.  On the Granularity and Clustering of Directed Acyclic Task Graphs , 1993, IEEE Trans. Parallel Distributed Syst..

[32]  Ann L. Chervenak,et al.  Characterizing and profiling scientific workflows , 2013, Future Gener. Comput. Syst..

[33]  Fang Dong,et al.  Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints , 2016, Cluster Computing.

[34]  Albert Y. Zomaya,et al.  Resource-efficient workflow scheduling in clouds , 2015, Knowl. Based Syst..

[35]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[36]  Rajkumar Buyya,et al.  A Particle Swarm Optimization-Based Heuristic for Scheduling Workflow Applications in Cloud Computing Environments , 2010, 2010 24th IEEE International Conference on Advanced Information Networking and Applications.

[37]  Daniel S. Katz,et al.  Montage: a grid-enabled engine for delivering custom science-grade mosaics on demand , 2004, SPIE Astronomical Telescopes + Instrumentation.

[38]  Sai Peck Lee,et al.  Cost optimization approaches for scientific workflow scheduling in cloud and grid computing: A review, classifications, and open issues , 2016, J. Syst. Softw..

[39]  Cees T. A. M. de Laat,et al.  An agent based network resource planner for workflow applications , 2011, Multiagent Grid Syst..

[40]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[41]  Cees T. A. M. de Laat,et al.  A Software Workbench for Interactive, Time Critical and Highly Self-Adaptive Cloud Applications (SWITCH) , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[42]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[43]  José Antonio Lozano,et al.  A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments , 2014, Journal of Grid Computing.