Efficient Workflow Scheduling for Grid Computing Using a Leveled Multi-objective Genetic Algorithm

Task scheduling and resource allocation are important problems in grid computing. The workflow management system receives inter-dependent tasks from the users and allocates each task to an appropriate resource based on user requirements and constraints such as budget and deadline. Thus this system has a significant effect on performance and the efficient use of resources. In general, mapping tasks to distributed resources is an NP-hard problem. Hence, heuristic and meta-heuristic methods are typically employed. Moreover, since tasks can enter the system at any time, the task scheduling runtime is an important parameter for workflow management systems. This paper presents a fast method for scheduling workflows in a grid environment based on a multi-objective Genetic Algorithm (GA). In the proposed method, the workflows and chromosomes in the GA are assigned to levels to reduce the scheduling time. In addition, the proposed method prevents infeasible solutions being produced in new generations, so task dependencies do not need to be checked. New crossover and mutation operators are proposed to improve convergence and maintain solution diversity. Experimental results are presented and evaluated using several well-known metrics as well as a new metric. This shows the effectiveness of the proposed method compared to other approaches.

[1]  Ritu Garg,et al.  Multi-Objective Optimization to Workflow Grid Scheduling using Reference Point based Evolutionary Algorithm , 2011 .

[2]  Gregor von Laszewski,et al.  QoS guided Min-Min heuristic for grid task scheduling , 2003, Journal of Computer Science and Technology.

[3]  Radu Prodan,et al.  ASKALON: a tool set for cluster and Grid computing , 2005, Concurr. Pract. Exp..

[4]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Dinkar N. Bhat An evolutionary measure for image matching , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[7]  Gary G. Yen,et al.  Dynamic multiobjective evolutionary algorithm: adaptive cell-based rank and density estimation , 2003, IEEE Trans. Evol. Comput..

[8]  Edward A. Lee,et al.  Scientific workflow management and the Kepler system , 2006, Concurr. Comput. Pract. Exp..

[9]  Rajkumar Buyya,et al.  Multi-objective planning for workflow execution on Grids , 2007, 2007 8th IEEE/ACM International Conference on Grid Computing.

[10]  Helen D. Karatza,et al.  Multi-Criteria Job Scheduling in Grid Using an Accelerated Genetic Algorithm , 2012, Journal of Grid Computing.

[11]  Aravind Seshadri,et al.  A FAST ELITIST MULTIOBJECTIVE GENETIC ALGORITHM: NSGA-II , 2000 .

[12]  Radu Prodan,et al.  Towards a general model of the multi-criteria workflow scheduling on the grid , 2009, Future Gener. Comput. Syst..

[13]  Daniel S. Katz,et al.  Pegasus: A framework for mapping complex scientific workflows onto distributed systems , 2005, Sci. Program..

[14]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[15]  Nawwaf N. Kharma,et al.  A high performance algorithm for static task scheduling in heterogeneous distributed computing systems , 2008, J. Parallel Distributed Comput..

[16]  Adam Arbree,et al.  Mapping Abstract Complex Workflows onto Grid Environments , 2003, Journal of Grid Computing.

[17]  Dick H. J. Epema,et al.  Cost-driven scheduling of grid workflows using Partial Critical Paths , 2010, 2010 11th IEEE/ACM International Conference on Grid Computing.

[18]  Rajkumar Buyya,et al.  A Taxonomy of Workflow Management Systems for Grid Computing , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[19]  Fahime Moein-darbari,et al.  Scheduling of scientific workflows using a chaos-genetic algorithm , 2010, ICCS.

[20]  Rajkumar Buyya,et al.  Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms , 2006, Sci. Program..

[21]  I. Foster,et al.  The Physiology of the Grid , 2003 .

[22]  Radu Prodan,et al.  Taxonomies of the Multi-Criteria Grid Workflow Scheduling Problem , 2008 .

[23]  David W. Coit,et al.  Multi-objective optimization using genetic algorithms: A tutorial , 2006, Reliab. Eng. Syst. Saf..

[24]  Fei Cao,et al.  Distributed workflow mapping algorithm for maximized reliability under end-to-end delay constraint , 2013, The Journal of Supercomputing.

[25]  Qian Tao,et al.  A Grid Workflow Scheduling Optimization Approach for e-Business Application , 2010, 2010 International Conference on E-Business and E-Government.

[26]  David Corne,et al.  The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimisation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[27]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[28]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[29]  Francine Berman,et al.  New Grid Scheduling and Rescheduling Methods in the GrADS Project , 2004, IPDPS Next Generation Software Program - NSFNGS - PI Workshop.

[30]  Andrei Tchernykh,et al.  Multiple Workflow Scheduling Strategies with User Run Time Estimates on a Grid , 2012, Journal of Grid Computing.

[31]  Jan Broeckhove,et al.  Runtime Prediction Based Grid Scheduling of Parameter Sweep Jobs , 2008, 2008 IEEE Asia-Pacific Services Computing Conference.

[32]  Marco Laumanns,et al.  SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization , 2002 .

[33]  Carl Kesselman,et al.  Optimizing Grid-Based Workflow Execution , 2005, Journal of Grid Computing.

[34]  Nawwaf N. Kharma,et al.  GATS 1.0: a novel GA-based scheduling algorithm for task scheduling on heterogeneous processor nets , 2005, GECCO '05.

[35]  Radu Prodan,et al.  Negotiation-Based Scheduling of Scientific Grid Workflows Through Advance Reservations , 2010, Journal of Grid Computing.

[36]  Radu Prodan,et al.  Scheduling of scientific workflows in the ASKALON grid environment , 2005, SGMD.

[37]  Rajkumar Buyya,et al.  Multiobjective differential evolution for scheduling workflow applications on global Grids , 2009 .

[38]  R. K. Ursem Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[39]  Lothar Thiele,et al.  A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers , 2006 .

[40]  Rajkumar Buyya,et al.  Market-oriented Grids and Utility Computing: The State-of-the-art and Future Directions , 2008, Journal of Grid Computing.

[41]  Sang Cheol Kim,et al.  Push-pull: guided search DAG scheduling for heterogeneous clusters , 2005, 2005 International Conference on Parallel Processing (ICPP'05).

[42]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[43]  Rainer Schmidt,et al.  VGE - a service-oriented grid environment for on-demand supercomputing , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[44]  Jian-Gang Yang,et al.  A genetic algorithm for tasks scheduling in parallel multiprocessor systems , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[45]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[46]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[47]  Matthew R. Pocock,et al.  Taverna: a tool for the composition and enactment of bioinformatics workflows , 2004, Bioinform..

[48]  Mei-Hui Su,et al.  Characterization of scientific workflows , 2008, 2008 Third Workshop on Workflows in Support of Large-Scale Science.

[49]  Joachim Geiler,et al.  Workflow-based Grid applications , 2006, Future Gener. Comput. Syst..

[50]  Hai Jin,et al.  Dependable Grid Workflow Scheduling Based on Resource Availability , 2012, Journal of Grid Computing.

[51]  Hong Linh Truong,et al.  ASKALON: a tool set for cluster and Grid computing: Research Articles , 2005 .

[52]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[53]  R. Buyya,et al.  A budget constrained scheduling of workflow applications on utility Grids using genetic algorithms , 2006, 2006 Workshop on Workflows in Support of Large-Scale Science.

[54]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.