A market-oriented hierarchical scheduling strategy in cloud workflow systems

A cloud workflow system is a type of platform service which facilitates the automation of distributed applications based on the novel cloud infrastructure. One of the most important aspects which differentiate a cloud workflow system from its other counterparts is the market-oriented business model. This is a significant innovation which brings many challenges to conventional workflow scheduling strategies. To investigate such an issue, this paper proposes a market-oriented hierarchical scheduling strategy in cloud workflow systems. Specifically, the service-level scheduling deals with the Task-to-Service assignment where tasks of individual workflow instances are mapped to cloud services in the global cloud markets based on their functional and non-functional QoS requirements; the task-level scheduling deals with the optimisation of the Task-to-VM (virtual machine) assignment in local cloud data centres where the overall running cost of cloud workflow systems will be minimised given the satisfaction of QoS constraints for individual tasks. Based on our hierarchical scheduling strategy, a package based random scheduling algorithm is presented as the candidate service-level scheduling algorithm and three representative metaheuristic based scheduling algorithms including genetic algorithm (GA), ant colony optimisation (ACO), and particle swarm optimisation (PSO) are adapted, implemented and analysed as the candidate task-level scheduling algorithms. The hierarchical scheduling strategy is being implemented in our SwinDeW-C cloud workflow system and demonstrating satisfactory performance. Meanwhile, the experimental results show that the overall performance of ACO based scheduling algorithm is better than others on three basic measurements: the optimisation rate on makespan, the optimisation rate on cost and the CPU time.

[1]  Hector Garcia-Molina,et al.  Deadline Assignment in a Distributed Soft Real-Time System , 1997, IEEE Trans. Parallel Distributed Syst..

[2]  Jeffrey C. Carver.  First International Workshop on Software Engineering for Computational Science & Engineering , 2009, Computing in Science & Engineering.

[3]  Wil vanderAalst,et al.  Workflow Management: Models, Methods, and Systems , 2004 .

[4]  Dorothy Ndedi Monekosso,et al.  A review of ant algorithms , 2009, Expert Syst. Appl..

[5]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[6]  Ruay-Shiung Chang,et al.  An ant algorithm for balanced job scheduling in grids , 2009, Future Gener. Comput. Syst..

[7]  Hai Jin,et al.  DAGMap: efficient and dependable scheduling of DAG workflow job in Grid , 2010, The Journal of Supercomputing.

[8]  Jinjun Chen,et al.  A throughput maximization strategy for scheduling transaction-intensive workflows on SwinDeW-G , 2008 .

[9]  Xiao Liu,et al.  Handling Recoverable Temporal Violations in Scientific Workflow Systems: A Workflow Rescheduling Based Strategy , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[10]  Fatos Xhafa,et al.  Metaheuristics for scheduling in distributed computing environments , 2008 .

[11]  Xiao Liu,et al.  Achieving On-Time Delivery: A Two-Stage Probabilistic Scheduling Strategy for Software Projects , 2009, ICSP.

[12]  Thomas Erl,et al.  SOA Principles of Service Design , 2007 .

[13]  John Mylopoulos,et al.  Workflow Management Models , Methods , and Systems , 2002 .

[14]  Ian J. Taylor,et al.  Workflows and e-Science: An overview of workflow system features and capabilities , 2009, Future Gener. Comput. Syst..

[15]  Rajkumar Buyya,et al.  A Taxonomy of Workflow Management Systems for Grid Computing , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[16]  Rajkumar Buyya,et al.  Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms , 2006, Sci. Program..

[17]  Heon-Chang Yu,et al.  Adaptive service scheduling for workflow applications in Service-Oriented Grid , 2009, The Journal of Supercomputing.

[18]  R. Buyya,et al.  Market-Oriented Grid and Utility Computing , 2009 .

[19]  Chisu Wu,et al.  Genetic-algorithm-based real-time task scheduling with multiple goals , 2004, J. Syst. Softw..

[20]  黄亚明,et al.  NOAH , 2012 .

[21]  María Cristina Riff,et al.  Towards an immune system that solves CSP , 2007, 2007 IEEE Congress on Evolutionary Computation.

[22]  Jinjun Chen,et al.  Multiple states based temporal consistency for dynamic verification of fixed‐time constraints in Grid workflow systems , 2007, Concurr. Comput. Pract. Exp..

[23]  Xiao Liu,et al.  A Probabilistic Strategy for Setting Temporal Constraints in Scientific Workflows , 2008, BPM.

[24]  Hai Jin,et al.  ServiceFlow: QoS-based hybrid service-oriented grid workflow system , 2009, The Journal of Supercomputing.

[25]  Ladislau Bölöni,et al.  A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems , 2001, J. Parallel Distributed Comput..

[26]  Yun Yang,et al.  SwinDeW-a p2p-based decentralized workflow management system , 2006, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[27]  Averill M. Law,et al.  Simulation modelling and analysis , 1991 .

[28]  Jinjun Chen,et al.  Trust-based robust scheduling and runtime adaptation of scientific workflow , 2009 .

[29]  Michelle D. Moore,et al.  An accurate parallel genetic algorithm to schedule tasks on a cluster , 2004, Parallel Comput..

[30]  José Duato,et al.  A New Cost-Effective Technique for QoS Support in Clusters , 2007, IEEE Transactions on Parallel and Distributed Systems.

[31]  Y.-K. Kwok,et al.  Static scheduling algorithms for allocating directed task graphs to multiprocessors , 1999, CSUR.

[32]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[33]  Yang Zhang,et al.  Hybrid Re-scheduling Mechanisms for Workflow Applications on Multi-cluster Grid , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[34]  Pingzhi Fan,et al.  An efficient flow-shop scheduling algorithm based on a hybrid particle swarm optimization model , 2007, Expert Syst. Appl..

[35]  Lei Zhang,et al.  Task Scheduling Based on PSO Algorithm in Computational Grid , 2006, Sixth International Conference on Intelligent Systems Design and Applications.

[36]  Jun Zhang,et al.  An Ant Colony Optimization Approach to a Grid Workflow Scheduling Problem With Various QoS Requirements , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[37]  Jinjun Chen,et al.  Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems , 2007, TAAS.

[38]  Dennis Gannon,et al.  Workflows for e-Science, Scientific Workflows for Grids , 2014 .

[39]  Jun Zhang,et al.  A Novel Set-Based Particle Swarm Optimization Method for Discrete Optimization Problems , 2010, IEEE Transactions on Evolutionary Computation.

[40]  Jun Zhang,et al.  Workflow scheduling in grids: an ant colony optimization approach , 2007, 2007 IEEE Congress on Evolutionary Computation.

[41]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[42]  Yuan Yingchun.  Hybrid particle swarm optimization algorithm for cost minimization in service-workflows with due dates , 2008 .

[43]  Radu Prodan,et al.  Taxonomies of the Multi-Criteria Grid Workflow Scheduling Problem , 2008 .

[44]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[45]  Xiao Liu,et al.  An Algorithm in SwinDeW-C for Scheduling Transaction-Intensive Cost-Constrained Cloud Workflows , 2008, 2008 IEEE Fourth International Conference on eScience.

[46]  Yuan-Chun Jiang,et al.  Preventing Temporal Violations in Scientific Workflows: Where and How , 2011, IEEE Transactions on Software Engineering.

[47]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[48]  Akhil Sahai,et al.  Towards Automated SLA Management for Web Services , 2002 .

[49]  YangYun,et al.  Temporal dependency-based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems , 2011 .

[50]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[51]  Hai Jin,et al.  Peer-to-Peer Based Grid Workflow Runtime Environment of SwinDeW-G , 2007, Third IEEE International Conference on e-Science and Grid Computing (e-Science 2007).

[52]  Danilo Ardagna,et al.  Adaptive Service Composition in Flexible Processes , 2007, IEEE Transactions on Software Engineering.

[53]  Sriram Ramabhadran,et al.  Cloud control with distributed rate limiting , 2007, SIGCOMM 2007.

[54]  Jinjun Chen,et al.  A Workflow Engine-Driven SOA-Based Cooperative Computing Paradigm in Grid Environments , 2008, Int. J. High Perform. Comput. Appl..

[55]  Rajkumar Buyya,et al.  Workflow scheduling algorithms for grid computing , 2008 .