New approach to allocation planning of many‐task workflows on clouds

Experience has shown that a priori created static resource allocation plans are vulnerable to runtime deviations and hence often become uneconomic or highly exceed a predefined soft deadline. The assumption of constant task execution times during allocation planning is even more unlikely in a cloud environment where virtualized resources vary in performance. Revising the initially created resource allocation plan at runtime allows the scheduler to react on deviations between planning and execution. Such an adaptive rescheduling of a many‐task application workflow is only feasible, when the planning time can be handled efficiently at runtime. In this paper, we present the static low‐complexity resource allocation planning algorithm (LCP) applicable to efficiently schedule many‐task scientific application workflows on cloud resources of different capabilities. The benefits of the presented algorithm are benchmarked against alternative approaches. The benchmark results show that LCP is not only able to compete against higher complexity algorithms in terms of planned costs and planned makespan but also outperforms them significantly by magnitudes of 2 to 160 in terms of required planning time. Hence, LCP is superior in terms of practical usability where low planning time is essential such as in our targeted online rescheduling scenario.

[1]  Andrei Tchernykh,et al.  Multiple Workflow Scheduling Strategies with User Run Time Estimates on a Grid , 2012, Journal of Grid Computing.

[2]  Ewa Deelman,et al.  Experiences using cloud computing for a scientific workflow application , 2011, ScienceCloud '11.

[3]  Denis A. Nasonov,et al.  Workflow Scheduling Algorithms for Hard-deadline Constrained Cloud Environments , 2016, ICCS.

[4]  Radu Prodan,et al.  Overhead Analysis of Scientific Workflows in Grid Environments , 2008, IEEE Transactions on Parallel and Distributed Systems.

[5]  Marios D. Dikaiakos,et al.  Scheduling Workflows with Budget Constraints , 2007, Grid 2007.

[6]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[7]  Ewa Deelman,et al.  The application of cloud computing to scientific workflows: a study of cost and performance , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[8]  Radu Prodan,et al.  Budget-Constrained Resource Provisioning for Scientific Applications in Clouds , 2013, 2013 IEEE 5th International Conference on Cloud Computing Technology and Science.

[9]  Lee Gillam,et al.  Towards Performance Prediction for Public Infrastructure Clouds: An EC2 Case Study , 2013, 2013 IEEE 5th International Conference on Cloud Computing Technology and Science.

[10]  Weisong Shi,et al.  An Adaptive Rescheduling Strategy for Grid Workflow Applications , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[11]  Mei-Hui Su,et al.  Characterization of scientific workflows , 2008, 2008 Third Workshop on Workflows in Support of Large-Scale Science.

[12]  R. V. van Nieuwpoort,et al.  The Grid 2: Blueprint for a New Computing Infrastructure , 2003 .

[13]  Matei Ripeanu,et al.  Amazon S3 for science grids: a viable solution? , 2008, DADC '08.

[14]  Marian Bubak,et al.  Cost Optimization of Execution of Multi-level Deadline-Constrained Scientific Workflows on Clouds , 2013, PPAM.

[15]  Alexandru Iosup,et al.  Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing , 2011, IEEE Transactions on Parallel and Distributed Systems.

[16]  Min Chen,et al.  Cost adaptive workflow scheduling in cloud computing , 2014, ICUIMC '14.

[17]  Rajkumar Buyya,et al.  Multi-objective planning for workflow execution on Grids , 2007, 2007 8th IEEE/ACM International Conference on Grid Computing.

[18]  Rizos Sakellariou,et al.  Balanced Task Clustering in Scientific Workflows , 2013, 2013 IEEE 9th International Conference on e-Science.

[19]  Uwe Schwiegelshohn,et al.  Bi-objective online scheduling with quality of service for IaaS clouds , 2014, 2014 IEEE 3rd International Conference on Cloud Networking (CloudNet).

[20]  Daniel S. Katz,et al.  Workflow task clustering for best effort systems with Pegasus , 2008, Mardi Gras Conference.

[21]  Uwe Schwiegelshohn,et al.  Energy-aware online scheduling: Ensuring quality of service for IaaS clouds , 2014, 2014 International Conference on High Performance Computing & Simulation (HPCS).

[22]  Dick H. J. Epema,et al.  Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds , 2013, Future Gener. Comput. Syst..

[23]  DeelmanEwa,et al.  Algorithms for cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds , 2015 .

[24]  Ramin Yahyapour,et al.  Optimal Negotiation of Service Level Agreements for Cloud-Based Services through Autonomous Agents , 2014, 2014 IEEE International Conference on Services Computing.

[25]  Rizos Sakellariou,et al.  Using imbalance metrics to optimize task clustering in scientific workflow executions , 2015, Future Gener. Comput. Syst..

[26]  G. Bruce Berriman,et al.  On the Use of Cloud Computing for Scientific Workflows , 2008, 2008 IEEE Fourth International Conference on eScience.

[27]  Ewa Deelman,et al.  Scientific workflows and clouds , 2010, ACM Crossroads.

[28]  Jin-Soo Kim,et al.  Cost optimized provisioning of elastic resources for application workflows , 2011, Future Gener. Comput. Syst..

[29]  Rizos Sakellariou,et al.  A low-cost rescheduling policy for efficient mapping of workflows on grid systems , 2004, Sci. Program..