Job Allocation Strategies with User Run Time Estimates for Online Scheduling in Hierarchical Grids

We address non-preemptive non-clairvoyant online scheduling of parallel jobs on a Grid. We consider a Grid scheduling model with two stages. At the first stage, jobs are allocated to a suitable Grid site, while at the second stage, local scheduling is independently applied to each site. We analyze allocation strategies depending on the type and amount of information they require. We conduct a comprehensive performance evaluation study using simulation and demonstrate that our strategies perform well with respect to several metrics that reflect both user- and system-centric goals. Unfortunately, user run time estimates and information on local schedules does not help to significantly improve the outcome of the allocation strategies. When examining the overall Grid performance based on real data, we determined that an appropriate distribution of job processor requirements over the Grid has a higher performance than an allocation of jobs based on user run time estimates and information on local schedules. In general, our experiments showed that rather simple schedulers with minimal information requirements can provide a good performance.

[1]  Uwe Schwiegelshohn,et al.  Online Hierarchical Job Scheduling on Grids , 2008 .

[2]  Uwe Schwiegelshohn,et al.  On an on-line scheduling problem for parallel jobs , 2002, Inf. Process. Lett..

[3]  Marco Aurélio Amaral Henriques,et al.  An Adaptive Scheduler for Grids , 2006, Journal of Grid Computing.

[4]  Dan Tsafrir,et al.  Modeling User Runtime Estimates , 2005, JSSPP.

[5]  E.L. Lawler,et al.  Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey , 1977 .

[6]  Ronald L. Graham,et al.  Bounds for Multiprocessor Scheduling with Resource Constraints , 1975, SIAM J. Comput..

[7]  Jarek Nabrzyski,et al.  A multicriteria approach to two-level hierarchy scheduling in grids , 2008, J. Sched..

[8]  Uwe Schwiegelshohn,et al.  On Advantages of Grid Computing for Parallel Job Scheduling , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[9]  Dan Tsafrir,et al.  Backfilling Using System-Generated Predictions Rather than User Runtime Estimates , 2007, IEEE Transactions on Parallel and Distributed Systems.

[10]  Helen D. Karatza,et al.  Resource Allocation Strategies in a 2-Level Hierarchical Grid System , 2008, 41st Annual Simulation Symposium (anss-41 2008).

[11]  Dmitry N. Zotkin,et al.  Job-length estimation and performance in backfilling schedulers , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[12]  Krzysztof Rzadca,et al.  Cooperation in Multi-organization Scheduling , 2009, Euro-Par.

[13]  Dror G. Feitelson,et al.  Utilization, Predictability, Workloads, and User Runtime Estimates in Scheduling the IBM SP2 with Backfilling , 2001, IEEE Trans. Parallel Distributed Syst..

[14]  Zdenek Salvet,et al.  The EU DataGrid Workload Management System: towards the second major release , 2003, ArXiv.

[15]  Jesús Labarta,et al.  eNANOS Grid Resource Broker , 2005, EGC.

[16]  Joseph Naor,et al.  On-Line Load Balancing in a Hierarchical Server Topology , 2002, SIAM J. Comput..

[17]  Sergey Zhuk.  Approximate algorithms to pack rectangles into several strips , 2006 .

[18]  Zoran Constantinescu,et al.  Advances in Grid Computing , 2011 .

[19]  Wolfgang Ziegler,et al.  A Meta-scheduling Service for Co-allocating Arbitrary Types of Resources , 2005, PPAM.

[20]  Uwe Schwiegelshohn,et al.  On-line hierarchical job scheduling on grids with admissible allocation , 2010, J. Sched..

[21]  Uwe Schwiegelshohn,et al.  Attributes for communication between Grid scheduling instances , 2004 .

[22]  Marco Vanneschi,et al.  From Grids To Service and Pervasive Computing , 2008 .

[23]  Kavitha Ranganathan,et al.  Simulation Studies of Computation and Data Scheduling Algorithms for Data Grids , 2003, Journal of Grid Computing.

[24]  Uwe Schwiegelshohn,et al.  A system-centric metric for the evaluation of online job schedules , 2011, J. Sched..

[25]  Dan Tsafrir,et al.  The Dynamics of Backfilling: Solving the Mystery of Why Increased Inaccuracy May Help , 2006, 2006 IEEE International Symposium on Workload Characterization.

[26]  Rajkumar Buyya,et al.  A taxonomy and survey of grid resource management systems for distributed computing , 2002, Softw. Pract. Exp..

[27]  Arutyun Avetisyan,et al.  Comparison of scheduling heuristics for grid resource broker , 2004, Proceedings of the Fifth Mexican International Conference in Computer Science, 2004. ENC 2004..

[28]  Jesús Labarta,et al.  Prediction f based models for evaluating backfilling scheduling policies , 2007 .

[29]  Youcef Derbal,et al.  Entropic Grid Scheduling , 2006, Journal of Grid Computing.

[30]  U. Schwiegelshohn.  MISTA 2009 An Owner-centric Metric for the Evaluation of Online Job Schedules , 2009 .

[31]  Jarek Nabrzyski,et al.  Grid resource management: state of the art and future trends , 2004 .

[32]  Mario Marchisio.  European Grid Conference (EGC) 2005 , 2005 .

[33]  Ramin Yahyapour,et al.  Applying economic scheduling methods to Grid environments , 2004 .

[34]  Andrea C. Arpaci-Dusseau,et al.  The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance , 2002, JSSPP.

[35]  José Luis Vázquez-Poletti,et al.  A comparison between two grid scheduling philosophies: EGEE WMS and Grid Way , 2007, Multiagent Grid Syst..

[36]  Gabor Terstyanszky,et al.  Extracting performance hints for grid users using data mining techniques: a case study in the NGS , 2007 .

[37]  Domenico Talia,et al.  Modeling and Supporting Grid Scheduling , 2008, Journal of Grid Computing.

[38]  Cynthia Bailey Lee,et al.  Are User Runtime Estimates Inherently Inaccurate? , 2004, JSSPP.

[39]  Warren Smith,et al.  Improving resource selection and scheduling using predictions , 2004 .

[40]  Ramin Yahyapour,et al.  Benefits of global grid computing for job scheduling , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[41]  Klaus Jansen,et al.  A Fast 5/2-Approximation Algorithm for Hierarchical Scheduling , 2010, Euro-Par.

[42]  Ivan Rodero,et al.  The Grid Backfilling: a Multi-Site Scheduling Architecture with Data Mining Prediction Techniques , 2008 .

[43]  Uwe Schwiegelshohn,et al.  Online scheduling in grids , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[44]  Johan Tordsson,et al.  An interoperable, standards-based grid resource broker and job submission service , 2005, First International Conference on e-Science and Grid Computing (e-Science'05).

[45]  Andrei Tchernykh,et al.  Two Level Job-Scheduling Strategies for a Computational Grid , 2005, PPAM.

[46]  Wolfgang Ziegler,et al.  Grid Middleware and Services , 2008 .

[47]  Layuan Li,et al.  Multi-level scheduling for global optimization in grid computing , 2008, Comput. Electr. Eng..

[48]  Dan Tsafrir,et al.  Session-Based, Estimation-less, and Information-less Runtime Prediction Algorithms for Parallel and Grid Job Scheduling , 2006 .